nanocoder-v1

NanoCoder V1 🧠⚡

A 4B parameter full-stack coding assistant fine-tuned from Qwen3-4B using Unsloth + LoRA. Trained through a multi-phase pipeline with joint domain training and validation-driven checkpoint selection.

Best checkpoint: step 150 — combined score 73.6% across all skill domains.

---

📊 Benchmarks

---

🎯 What it does well

• Backend — API design, auth (JWT/bcrypt), SQL/NoSQL, N+1 fixes, CORS
• Debugging agent — structured reasoning (### Thought → ### Action → ### Patch → ### Reasoning)
• Full-stack integration — connects frontend + backend flows
• Bug pattern recognition — race conditions, memory leaks, type errors

⚠️ Known limitations

• Frontend scores lower than backend (weakest domain in v1)
• Not a replacement for larger models (7B+) on hard competitive programming
• English-only

---

🚀 Usage

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="usernamebetter/nanocoder-v1",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

SYSTEM = "You are NanoCoder, an expert Senior Full-Stack Engineer and debugging agent."

prompt = (
    f"<|im_start|>system\n{SYSTEM}<|im_end|>\n"
    f"<|im_start|>user\nFix this React hydration error: useState(Date.now())<|im_end|>\n"
    f"<|im_start|>assistant\n"
)

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=300, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

---

🏗️ Training pipeline

Multi-phase joint training from Qwen3-4B base:

1. Phase 1 — General coding (Magicoder-Evol-Instruct)
2. Phase 2 — Frontend specialization
3. Phase 3 — Fullstack (frontend + backend interleaved)
4. Phase 4 — Agent reasoning training
5. Final — Joint retrain from base with all domains mixed (this checkpoint)

Training configuration

• Base: Qwen3-4B (4-bit quantized)
• LoRA: r=32, alpha=32, dropout=0
• LR: 1e-5 with cosine scheduler
• Steps: 400 (best checkpoint at step 150)
• Batch: 2 × grad accum 4 = effective 8
• Optimizer: adamw_8bit
• Dataset: ~13.7k samples interleaved
  • 🤖 Agent (synthetic + real): 40%
  • 🎨 Frontend: 35%
  • ⚙️ Backend: 15%
  • 🐛 Bug fixing: 10%

Data sources

• ise-uiuc/Magicoder-Evol-Instruct-110K
• sahil2801/CodeAlpaca-20k
• nickrosh/Evol-Instruct-Code-80k-v1
• iamtarun/code_instructions_120k_alpaca
• m-a-p/CodeFeedback-Filtered-Instruction
• bigcode/self-oss-instruct-sc2-exec-filter-50k
• HuggingFaceH4/CodeAlpaca_20K
• TokenBender/code_instructions_122k_alpaca_style
• Custom synthetic agent examples with structured reasoning format

---

🧪 Prompt format

Uses Qwen chat template:

<|im_start|>system
You are NanoCoder, an expert Senior Full-Stack Engineer and debugging agent.
<|im_end|>
<|im_start|>user
{your question}
<|im_end|>
<|im_start|>assistant

For debugging tasks, the model responds in structured format:

### Thought:
{root cause analysis}

### Action:
{what to do}

### Patch:
{code fix}

### Reasoning:
{why it works}

---

📅 Roadmap

• ✅ v1: Joint multi-domain training (this release)
• 🚧 v2: Frontend boost + reasoning domain + label smoothing + cosine restarts
• 🚧 v3: DPO alignment + tool calling
• 🚧 GGUF: Q4_K_M / Q5_K_M / Q8_0 exports

---

🙏 Credits

• Base model: Qwen/Qwen3-4B
• Fine-tuning framework: Unsloth
• Training: Kaggle T4 + Google Colab T4

---

📄 License

Apache-2.0 (inherited from Qwen3-4B base).