Nanbeige4.1-3B-EE-Reasoning-SFT

Nanbeige4.1-3B-EE-Reasoning-SFT

A LoRA fine-tune of Nanbeige4.1-3B trained on electrical engineering Q&A data with the goal of reducing reasoning trace length and producing more concise, emoji-free answers.

This is my first end-to-end LLM fine-tuning experiment. Results and analysis are documented below.

---

Training Pipeline

The full pipeline consists of 7 scripts, each handling a distinct stage:

1. Data Curation — Parses Stack Exchange Electronics XML dumps, filters by upvotes and quality thresholds, producing clean Q&A pairs.
2. Data Enrichment — Uses a teacher LLM to expand and improve the raw Stack Exchange answers into richer technical explanations.
3. Trace Generation — Uses a teacher LLM to synthesise shortened reasoning traces, targeting ~2K tokens versus the base model's ~8K average.
4. Data Splitting — Splits the dataset into training and evaluation sets.
5. Fine-Tuning — QLoRA training via Unsloth with sequence packing and response-only training.
6. Merge & Convert — Merges the LoRA adapter into the base model and quantizes to GGUF format.
7. Benchmarking — Compares base and fine-tuned models on GSM8K, MMLU Electrical Engineering, and held-out EE Q&A pairs. Uses exact-match accuracy for GSM8K and MMLU-EE, and LLM-as-judge win rate for EE-QA.

---

Results

The behavioural objective was achieved — reasoning traces and answer length were both significantly reduced:

However, LLM-as-judge evaluation on 50 held-out EE Q&A pairs showed significant quality degradation:

Why this happened: Nanbeige4.1-3B was trained with reinforcement learning, which shaped a specific internal search mechanism that the model relies on to retrieve and reason over its knowledge. Applying SFT with a small dataset of 700 examples was too rigid — it disrupted this mechanism rather than refining it, causing the model to produce shorter but less accurate responses. The model appears to need its extended reasoning process to access knowledge correctly, and that process cannot be easily compressed via SFT alone with limited data.

Full pipeline and methodology in the GitHub repository.

---

Usage

Option 1: LoRA Adapter (with Unsloth)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="amoughnieh/Nanbeige4.1-3B-EE-Reasoning-SFT",
    max_seq_length=8192,
    load_in_4bit=True,
)

system_prompt = "You are an expert electrical engineer. Answer technical questions accurately and in depth."

Option 2: GGUF (with Ollama or LM Studio)

The quantized GGUF file Nanbeige4.1-3B-EE-FT-r11-Q6_K.gguf is included in this repo and can be loaded directly in Ollama or LM Studio.

---

Training Details

Dataset

• Source: Stack Exchange Electronics data dump
• Quality filters: Minimum 5 upvotes, downvote ratio ≤ 30%, minimum 300 characters
• Training examples: 680
• Evaluation examples: 10
• Data enrichment: Raw answers enriched via Qwen-MT-Flash teacher model
• Trace generation: Shortened reasoning traces synthesised via Qwen-MT-Flash, targeting ~2K tokens

LoRA Configuration

Training Configuration

---

References

• Base Model: Nanbeige4.1-3B
• Paper: Nanbeige4.1 Technical Report
• Dataset: Stack Exchange Electronics
• Fine-tuning Framework: Unsloth
• Pipeline Code: GitHub Repository