Downloads · 30 days
7
14% of all-time downloads
antoninocutri/zagreus-italic-ikora-dpo-v3
zagreus-italic-ikora-dpo-v3 is a machine learning model from antoninocutri. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Zagreus 0.4B Italic Ikora DPO v3 is an Italian multiple-choice model fine-tuned from mii-llm/zagreus-0.4B-ita, developed for the mii-llm/Post-Training-Challenge.
Downloads · 30 days
7
14% of all-time downloads
All-time downloads
49
Public
Parameters
438M
893 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors876 MB · 98%
From the Hugging Face model README
Zagreus 0.4B Italic Ikora DPO v3 is an Italian multiple-choice model
fine-tuned from mii-llm/zagreus-0.4B-ita,
developed for the mii-llm/Post-Training-Challenge.
Not intended as a general-purpose assistant.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "antoninocutri/zagreus-italic-ikora-dpo-v3"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).to(device)
messages = [
{"role": "system", "content": "Sei un assistente utile."},
{
"role": "user",
"content": """Rispondi alla seguente domanda a scelta multipla sull'argomento 'geografia'. La tua risposta deve essere nel seguente formato: 'LETTERA' (senza virgolette) dove LETTERA è una tra ABCD. Scrivi solo la lettera corrispondente alla tua risposta senza spiegazioni.
Qual è la capitale d'Italia?
A) Roma
B) Milano
C) Torino
D) Napoli
Risposta:""",
},
]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(device)
output = model.generate(**inputs, max_new_tokens=8, do_sample=False)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
Note: this checkpoint uses the base model's tokenizer with a custom fallback chat template (the base is not natively instruction-tuned). Results are not guaranteed to reproduce with a different prompt or chat template.
Same architecture as the base model, mii-llm/zagreus-0.4B-ita
(~0.4B parameter Llama-style decoder-only transformer). Tokenizer/vocabulary
unchanged from the base model.
| Dataset | Contribution |
|---|---|
sapienzanlp/mmlu_italian | Italian MMLU translation, academic-knowledge MCQs |
s-conia/mmlu_italian | Independent Italian MMLU translation, same domain |
facebook/belebele (ita_Latn) | Native Italian reading comprehension MCQs |
cambridgeltl/xcopa (it) | Native Italian causal commonsense reasoning |
FinancialSupport/italic_sft | ITALIC-style Italian MCQs across multiple domains |
FinancialSupport/italic_sft_ext | Extended ITALIC-style Italian MCQs |
FinancialSupport/quiz_militare | Italian civic and general-knowledge MCQs |
efederici/pinocchio | Italian MCQs across culture, law, logic, science, foreign language |
All sources deduplicated cross-dataset (exact + Jaccard similarity on question text): 62,803 unique base questions → 55,716 after deduplication. Each question was expanded into balanced permutations of the answer options (2 to 5 options depending on source). For the DPO stage, the same questions were used to build within-question preference pairs (same content, different option order; the rejected letter is prioritized among the more common "attractive" positions to directly counter positional bias).
No ITALIC benchmark data or labels were used for training.
Two-stage training, both starting from the official base checkpoint:
Stage 1 — SFT
| Parameter | Value |
|---|---|
| Epochs | 1 |
| Learning rate | 3e-4 |
| Schedule | cosine decay with warmup |
| Batch size | 16 |
| Loss | completion-only (assistant tokens only) |
Stage 2 — DPO (this checkpoint)
| Parameter | Value |
|---|---|
| Epochs | 0.4 (partial) |
| Learning rate | 5e-6 |
| Beta | 0.10 |
| Batch size | 16 (grad_accum=2) |
| Pairs | within-question (same content, different option order) |
Hardware: 1x NVIDIA H100.
Evaluated with a custom transformers-based script replicating the official
ITALIC fast (no-CoT, 5-shot) prompt format and answer extraction. Not the
official vLLM harness — treat as a proxy; absolute numbers may not be
directly comparable to submissions evaluated with the official pipeline.
| Checkpoint | Full eval (10,000 Q) | Predicted letter distribution |
|---|---|---|
| Base model (no fine-tuning) | ~0.24–0.29 (quick eval only) | strongly A/B skewed |
| SFT only (same data, no DPO) | 0.3685 | A: 6728, B: 2418, C: 750, D: 94, E: 10 |
| This checkpoint (SFT + partial DPO) | 0.3730 | A: 6493, B: 2515, C: 856, D: 122, E: 14 |
Adding a short DPO stage on top of the SFT checkpoint gave a small but consistent accuracy improvement together with a modest reduction in positional bias (fewer A predictions, more mass on B/C/D/E). The predicted distribution remains skewed toward option A.
Full pipeline (data generation, training, evaluation scripts): zagreus-italic-ikora-sft-v3