Downloads · 30 days
2
7% of all-time downloads
RichardErkhov/MEscriva_-_ECE-PRYMMAL-0.5B-FT-EnhancedMUSREnsembleV3-awq
MEscriva_-_ECE-PRYMMAL-0.5B-FT-EnhancedMUSREnsembleV3-awq is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
2
7% of all-time downloads
All-time downloads
30
Public
Parameters
494M
470 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors458 MB · 97%
How the weights are stored.
I32358M · 72%
From the Hugging Face model README
Quantization made by Richard Erkhov.
ECE-PRYMMAL-0.5B-FT-EnhancedMUSREnsembleV3 - AWQ
library_name: transformers tags:
Ce modèle est une version hautement optimisée de Qwen-0.5B, spécialement conçue pour exceller dans le raisonnement multi-source (MUSR). Il représente la troisième version de notre architecture d'ensemble améliorée, atteignant des performances exceptionnelles sur le benchmark MUSR.
Tested on 500 samples from QASC validation set:
Confidence Distribution:
1.1 : 95.8%
Ce modèle est optimisé pour :
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("matouLeLoup/ECE-PRYMMAL-0.5B-FT-EnhancedMUSREnsembleV3")
tokenizer = AutoTokenizer.from_pretrained("matouLeLoup/ECE-PRYMMAL-0.5B-FT-EnhancedMUSREnsembleV3")
# Format de prompt optimal
prompt = f"""Context:
Fact 1: {fact1}
Fact 2: {fact2}
Question: {question}
Choices:
{choices}
Instructions:
1. Analyze both facts carefully
2. Connect the information
3. Choose the letter (A-H) that best answers the question
4. Explain your reasoning
Reasoned Answer:"""
# Génération
inputs = tokenizer(prompt, return_tensors="pt").to(device)
outputs = model.generate(
**inputs,
max_new_tokens=150,
num_beams=5,
temperature=0.6,
no_repeat_ngram_size=3
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
# Training details
Training Procedure
Training Hyperparameters
Learning rate: 2e-5
Batch size: 32
Weight decay: 0.1
Warmup steps: 0
Scheduler: polynomial
Training regime: bf16 mixed precision
# Evaluation Procedure
Tested on 500 random samples from QASC validation set
Evaluated for accuracy, confidence, and source usage
Detailed analysis of reasoning steps and response quality
# Limitations and Bias
Optimisé spécifiquement pour le format MUSR
Nécessite une structuration précise des prompts
Conçu pour des questions à choix multiples avec raisonnement
# Technical Specifications
Base model: Qwen-0.5B
Enhanced with optimized generation parameters
Uses letter-based answer format (A-H)
# Generation config
generation_config = {
"max_new_tokens": 150,
"num_beams": 5,
"temperature": 0.6,
"do_sample": False,
"length_penalty": 1.0,
"no_repeat_ngram_size": 3
}
@misc{PRYMMAL-EnhancedMUSREnsembleV3,
author = {matouLeLoup},
title = {ECE-PRYMMAL-0.5B-FT-EnhancedMUSREnsembleV3},
year = {2024},
publisher = {Hugging Face},
journal = {Hugging Face Hub},
howpublished = {\url{https://huggingface.co/matouLeLoup/ECE-PRYMMAL-0.5B-FT-EnhancedMUSREnsembleV3}}
}