qwen14b-code-trainer-v8_mixed

qwen14b-code-trainer-v8_mixed

LoRA adapter for Qwen/Qwen2.5-Coder-14B-Instruct, fine-tuned on the code-trainer-v8-mixed dataset. This is the V8 release — a corrective iteration over V7 that fixes multilingual hallucination and aligns tool-call formatting with Qwen2.5's native template.

Part of the Code-Trainer / RTPI pipeline (GitHub).

Status: the current Phase 5 GGUF conversion source. Superseded for training by V9, which improves <tool_call> tag emission fidelity.

What changed from V7

1. Native Qwen2.5 tool-call format — all tool-calling examples are formatted via tokenizer.apply_chat_template(tools=...) to produce the exact token sequence the model expects natively. V7 used Hermes-style <tool_call> tags tokenized as regular text, which the model learned inconsistently.
2. English-only filtering — a < 5% non-ASCII threshold on all slices eliminates the multilingual garbage tokens V7 emitted at sequence boundaries.
3. Added Slice D — 8K English instruction-following examples from OpenHermes-2.5 to anchor the model's language and prevent distributional drift toward tool-calling-only behaviour.
4. Switched Slice B source — from NousResearch/hermes-function-calling-v1 (Hermes format) to glaiveai/glaive-function-calling-v2 (cleaner, single-source, easier to reformat into native Qwen2.5 template).

Training data

• Dataset: cmndcntrlcyber/code-trainer-v8-mixed
• Total: 34,104 train / 3,789 validation (90/10 split, seed 42)

• Tool coverage: 19,651 / 34,104 train rows (57.6%) contain tool definitions
• Format: Unified ChatML, tool calls formatted via apply_chat_template(tools=...)

Training procedure

Key changes from V7:

• learning_rate: 1.0e-4 (was 1.5e-4) — even gentler to preserve base capabilities
• max_seq_length: 4096 (was 8192) — reduced to fix A100 OOM; agent prompts fit at 4K after truncation
• batch_size: 1 (was 2) — compensated by gradient accumulation 16

Evaluation

• HF Job: 6a73e7e66b79c09949c23c7e (status: CANCELED — adapter was pushed to Hub before the job was killed by HF Skills timeout enforcement)
• Fixes confirmed: multilingual hallucination eliminated; tool-call JSON emitted in native Qwen2.5 format.

Comparison across versions

Known issues (fixed in V9)

1. <tool_call> tag omission — the model outputs correct tool name and JSON arguments but sometimes omits the <tool_call> / </tool_call> wrapper tags that Ollama needs to parse structured tool_calls.
2. Trailing garbage after tool calls — some assistant turns include text after </tool_call>, weakening the stop signal.
3. Occasional tool-name lowercasing — ls instead of LS.

Root cause: <tool_call> tags are tokenized as multi-token sequences; Q4_K_M quantization loses some fidelity on these. The JSON payload pattern is learned strongly, the XML wrapper weakly.

How to use

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen2.5-Coder-14B-Instruct"
adapter_id = "cmndcntrlcyber/qwen14b-code-trainer-v8_mixed"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id, dtype=torch.bfloat16, device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

messages = [
    {"role": "system", "content": "You are a coding assistant with tool access."},
    {"role": "user", "content": "Read the file main.py and summarise its structure."},
]
inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True,
).to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Reproducibility

• Code: github.com/cmndcntrlcyber/code-trainer-pipeline
• Dataset build:

  python -m src.phase2_preprocessing.scripts.build_v8_mixed_dataset \
      --config src/config/config.yaml
  

• Training launch:

  python -m src.phase4_qwen_finetuning.scripts.launch_validation_sweep \
      --config src/config/config.yaml --only v8_mixed --wait
  

• W&B project: rtpi-phase4-qwen14b