codek-qwen2.5-coder-7b-lora-v2

CodeK LoRA v1 -- Qwen2.5-Coder-7B-Instruct

A LoRA adapter fine-tuned on the CodeK v2 dataset: a reasoning-first, pedagogical coding dataset with ~2x the seeds of v0. Teaches decomposition, bug diagnosis, contrast reasoning, and hypothesis-driven thinking about code.

Eval Results (Pass 2 ground-truth, 50 seeds)

The regression gap vs base model closed from -6% (v0) to -2% (v1). Evaluated on the same 50 seeds as v0 for direct comparison.

Note: best checkpoint (step 900, eval loss 0.0664) was rotated out during training (save_total_limit=3). checkpoint-1300 (eval loss 0.077) used instead. True best checkpoint would likely score 62–64%.

v0 → v1 Comparison

Training

Eval Loss Curve

Dataset

mechramc/codek-v2 (coming soon) -- 398 seeds, 4 augmentation passes, 5,075 ShareGPT pairs. Categories: data structures, algorithms, ML fundamentals, NN components, training infra, utilities, numerical, parsing.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")
model = PeftModel.from_pretrained(base, "mechramc/codek-qwen2.5-coder-7b-lora-v2")
tokenizer = AutoTokenizer.from_pretrained("mechramc/codek-qwen2.5-coder-7b-lora-v2")

Links

• v0 adapter (baseline)