chess-coach-32b-v6-dpo

Chess Coach 32B v6-dpo (earlier DPO preference adapter)

A LoRA adapter that DPO-sharpens the v4 SFT base on tier-appropriate move selection (the project's one trained behavior), trained on the deep-verified v6 preference pairs. This is the earlier DPO adapter: the live-served coach is now chess-coach-32b-v6-dpo2, the best-DPO refinement that supersedes this adapter. The v4 SFT base (chess-coach-32b-v4-qlora) remains the full-reproducible-eval reference and the fallback endpoint.

Model family and roles

• chess-coach-32b-v4-qlora: the SFT base, the full-reproducible-eval reference, and the fallback endpoint.
• chess-coach-32b-v6-dpo2: the live-served coach on the demo (Modal chess-coach-v6dpo2-4bit-maia, tuned:true), the best-DPO refinement of the v4 SFT base.
• chess-coach-32b-v6-dpo: the earlier DPO adapter, superseded by v6-dpo2.
• chess-coach-32b-v6-distill: the behavior-in-weights research adapter, evaluated with no grounding (0.325 tier-policy match).

What it is

• Base: unsloth/Qwen3-32B-unsloth-bnb-4bit (the same 4-bit base as v4).
• Init: a two-adapter TRL DPO setup where the policy and the frozen reference both initialize from the v4 LoRA, so the KL pressure holds soundness and format in place while DPO moves only the move choice (improve v4 without regressing it).
• LoRA: r=32, alpha=32, dropout=0 on q,k,v,o,gate,up,down (inherited from v4).

Preference data

Built from chess-coach-v6. For each pair the chosen response recommends the deep-verified tier-appropriate move (provenance.canonical_uci) and the rejected response is the same text with only the move swapped to the off-tier or off-spec contrast move (provenance.dpo_rejected_uci). Because chosen and rejected are byte-identical except for the move, DPO learns move preference, not prose style (840 pairs, 280 per tier).

Honest result (corrected v6 benchmark, 120 held-out TEST, grounded)

Re-scored in one controlled session against the deep-verified v6 labels, same strict extractor as the v4 reference eval (RESULTS_STAGE4_CORRECTED.md):

The preference tune sharpens the moat with no regression: +0.0195 overall tier-policy match, and the entire gain is the intermediate tier (0.808 vs 0.750, out of distribution). Beginner, advanced, soundness, and distinct-moves are unchanged from v4. v6-dpo2 (checkpoint step 200) later reached 0.892 overall and supersedes this adapter. See the honest limits below.

Honest limits

• The gain is confined to the intermediate tier; beginner and advanced already sit at their ceiling under grounding.
• Tier-policy match is agreement with the project's select_tier_move rule, which is a learnability metric, not certified best teaching.
• Grounding is required at inference. Without the per-tier Maia signal in the prompt the three tiers collapse to a single move; the behavior does not live in the weights.

Use

Load the base 4-bit model and apply this adapter with PEFT (the tokenizer is the base Qwen3 tokenizer). Run it with engine grounding in the prompt, the same move-review format v4 was trained on.

Links

License Apache-2.0, inherited from Qwen3-32B. The preference data is CC-BY-NC-4.0.