qwen14b-code-trainer-v9_mixed

qwen14b-code-trainer-v9_mixed

LoRA adapter for Qwen/Qwen2.5-Coder-14B-Instruct, fine-tuned on the code-trainer-v9-mixed dataset with a two-phase curriculum. This is the V9 release — focused on fixing <tool_call> tag emission so the model produces the exact wrapper tags Ollama needs to parse structured tool calls.

Part of the Code-Trainer / RTPI pipeline (GitHub).

Status: superseded for deployment by V10 (GRPO). V9 remains the SFT foundation — the V10 adapter chain merges V9 before applying GRPO reinforcement learning.

What changed from V8

1. Increased tool-calling density — Slice B expanded from 12K to 19K rows (tool-calling share: 31% → ~45% of the dataset).
2. Synthetic multi-tool-call examples — ~2K synthetic examples with 2–3 tool calls per assistant turn, teaching the model sequential tool emission.
3. Stop-after-tag cleanup — all assistant turns containing </tool_call> are stripped of trailing text, teaching a clean stop signal.
4. Tag completeness validation — records with unmatched <tool_call> / </tool_call> tags are dropped during dataset build.
5. Two-phase curriculum training — Phase A (80% of steps) trains on the full mixed dataset; Phase B (20% of steps) trains on the tool-calling-only subset at a higher learning rate for a final "polish" pass on tag formatting.
6. Q5_K_M default — the deployment quantization is bumped from Q4_K_M to Q5_K_M for improved tag-pattern fidelity (~1.5 GB larger, still fits RTX 5060 Ti 16GB).

Training data

• Dataset: cmndcntrlcyber/code-trainer-v9-mixed
• Total: 40,401 train / 4,489 validation (90/10 split, seed 42)

• Tool coverage: 25,964 / 40,401 train rows (64.3%) contain tool definitions
• Format: Unified ChatML, tool calls formatted via apply_chat_template(tools=...)

Training procedure

Phase A — Full mixed dataset (80% of total steps)

Phase B — Tool-calling polish (20% of total steps)

Evaluation

Version comparison

Intended use

• Direct use: load the adapter on top of Qwen/Qwen2.5-Coder-14B-Instruct for instruction-following code generation, tool calling, and multi-turn agent behaviour.
• Downstream: merge into the base model and quantize to Q5_K_M GGUF for local serving via llama.cpp, Ollama, or LM Studio.
• Out of scope: this adapter was not trained for safety alignment, RLHF, or non-code tasks.

Deployment notes

• Recommended quant: Q5_K_M (preserves multi-token <tool_call> tag patterns better than Q4_K_M). Q4_K_M is available as a fallback.
• Context length: 8,192 tokens recommended; the model was trained at 4,096 but the base model supports 32K.

How to use

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen2.5-Coder-14B-Instruct"
adapter_id = "cmndcntrlcyber/qwen14b-code-trainer-v9_mixed"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id, dtype=torch.bfloat16, device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

messages = [
    {"role": "system", "content": "You are a coding assistant with tool access."},
    {"role": "user", "content": "Read the file main.py and summarise its structure."},
]
inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True,
).to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Reproducibility

• Code: github.com/cmndcntrlcyber/code-trainer-pipeline
• Dataset build:

  python -m src.phase2_preprocessing.scripts.build_v9_mixed_dataset \
      --config src/config/config.yaml
  

• Training launch:

  python -m src.phase4_qwen_finetuning.scripts.launch_v9_training \
      --config src/config/config.yaml --wait
  

• W&B project: rtpi-phase4-qwen14b