Qwen2.5-32B-instruct-mix

Qwen2.5-32B-instruct-mix

A LoRA adapter for Qwen/Qwen2.5-32B from Teaching Qwen Why, a replication of Anthropic's "Teaching Claude Why" on open-weight models (code, data). This is the baseline: the base model trained on the instruction-tuning mix only. Used in Figures 1, 2, 3, 4, 5, 6, 7, 13, 14, 15 of the post (as the baseline).

Training

Each stage trains one LoRA adapter (rank 64, alpha 128, all linear layers; learning rate 1e-4, cosine schedule, effective batch 8 packed sequences), which is merged into the model before the next stage:

1. the 13K instruction-tuning mix, 2 epochs, loss on assistant turns only — adapter Qwen2.5-32B-instruct-mix (this repo)

Paths are in the data repository. Before the instruct-mix stage, the base model's untrained ChatML terminator rows are repaired (the graft, see the code repository); the patch is stored here as base_row_patch.safetensors.

Use

Serve the adapters in chain order on the grafted base with vLLM, on a GPU machine with the code repository:

RUN=model GRAFT=1 BASE_MODEL=Qwen/Qwen2.5-32B \
  ADAPTERS="SecondLookResearch/Qwen2.5-32B-instruct-mix" \
  bash evaluation/agentic_misalignment/serve_model.sh

serve_model.sh is in the code repository; it applies the graft, merges the adapters in order and starts the server.

Citation

@misc{wei2026teachingqwenwhy,
  title  = {Teaching Qwen Why},
  author = {Wei, Anastasia and Thompson, Jack and Qi, Brandon and Slocum, Stewart},
  year   = {2026},
  note   = {Second Look Research}
}