qwen3-8b-finetune-multit-mixed

Qwen3 8B — Italian Cultural Alignment [V3 — Mixed Training]

Qwen3 8B [V3] is a LoRA adapter fine-tuned on top of Qwen/Qwen3-8B to improve Italian cultural alignment using a hybrid training technique — interleaving thinking-format and non-thinking-format examples in a single SFT pass via ThinkingMode.MIXED. Trained on the Mult-IT dataset and evaluated on the ITALIC benchmark. V3 is the final version in the series and the only configuration where both inference modes improved simultaneously.

Author: Maruf Bepary, King's College London
Research report: Alignment in Large Language Models

---

Model Summary

---

Intended Use

This model is intended for:

• Italian language understanding — multiple-choice Q&A, cultural knowledge, and general instruction following in Italian. Both inference modes (No Thinking and Thinking) are fully functional.
• Research — studying hybrid training techniques for preserving dual-mode capabilities in reasoning language models; demonstrating that simultaneous improvement across both inference modes is achievable.
• Benchmarking — comparing Italian cultural alignment and reasoning performance across model sizes and training strategies.

Not recommended for:

• High-stakes or safety-critical applications.
• Languages other than Italian.
• Tasks requiring peak No Thinking performance only — V1 marginally outperforms V3 in No Thinking mode (73.77% vs 73.81%).

---

Key Finding — Dual-Mode Improvement

The hybrid training technique is the only configuration in the Qwen3 series where both inference modes improved simultaneously:

V1 improved No Thinking but collapsed Thinking (59.33%, −15.16 pp). V2 recovered Thinking (77.87%) but left No Thinking nearly unchanged (70.27%). V3 interleaves both formats in training, allowing the model to simultaneously retain chain-of-thought capability and improve Italian cultural knowledge.

---

Version Series

---

Training Details

LoRA Configuration

Training Hyperparameters

Framework & Hardware

Training Dataset — Mult-IT (Mixed Format)

• Dataset: Mult-IT — Multiple Choice Questions on Multiple Topics in Italian
• Source: CALAMITA Shared Task @ CLiC-it 2024
• Language: Italian
• Format: Single SFT pass with ThinkingMode.MIXED; thinking-format and non-thinking-format examples interleaved
• Reference: Mult-IT: Multiple Choice Questions on Multiple Topics in Italian (2024)

---

ITALIC Benchmark Results

Benchmark: ITALIC (NAACL 2025) — Italian Culture-Aware Natural Language Benchmark
Format: Zero-shot, multiple-choice (12 categories, 10,000 questions)
System prompt: "Sei un assistente utile."

No Thinking Mode — V3 vs Baseline

Baseline = Qwen3 8B (No Thinking), no fine-tuning.

Thinking Mode — V3 vs Baseline

Baseline = Qwen3 8B (Thinking), no fine-tuning.

Comparison with Other Models (No Thinking, ITALIC Total)

All scores evaluated under identical zero-shot conditions on the ITALIC benchmark.

---

Usage

Both thinking and no-thinking modes work well with this adapter.

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen3-8B"
adapter_id = "maruf-bepary/qwen3-8b-italian-v3-mixed"

# Load tokeniser and base model
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

messages = [
    {"role": "system", "content": "Sei un assistente utile."},
    {
        "role": "user",
        "content": (
            "Qual è la capitale d'Italia?\n"
            "A) Milano\nB) Roma\nC) Napoli\nD) Torino\n\n"
            "Rispondi con la lettera della risposta corretta."
        ),
    },
]

# Thinking mode — chain-of-thought is fully functional in V3
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=False,
    )

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)
print(response)

# No Thinking mode — also fully functional in V3
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=64,
        do_sample=False,
        temperature=None,
        top_p=None,
    )

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)
print(response)
# Expected output: "B"

---

Limitations

• Thinking mode in V3 (77.57%) is marginally below V2 (77.87%) — the mixed objective slightly dilutes peak thinking-mode gains relative to thinking-only training.
• No Thinking mode in V3 (73.81%) is only marginally above V1 (73.77%) — mixing adds negligible overhead for this mode.
• Morphology remains the weakest No Thinking category (52.86%), suggesting limited syntactic generalisation.
• Benchmark scope — evaluation was conducted solely on ITALIC; Italian cultural performance on other benchmarks (e.g. MMLU-IT, HellaSwag-IT) is unverified.
• Single-GPU training — training used one RTX 3090; larger batch sizes or multi-GPU configurations may yield different results.
• Dataset bias — Mult-IT is a multiple-choice dataset; the model may not generalise equally well to open-ended Italian generation tasks.

---

References

Related resources:

• Research report: Alignment in Large Language Models
• Base model: Qwen/Qwen3-8B
• ITALIC benchmark: RiTA-nlp/ITALIC
• Mult-IT dataset: sapienzanlp/Mult-IT
• PEFT documentation: huggingface.co/docs/peft