Affine-sn120-cand8-dpo-adapter

Affine SN120 — cand8 LoRA adapter (DPO on min(R,G))

The LoRA adapter only (27 MB). The merged full weights are at RLTT/Affine-sn120-cand8-dpo. Apply this to the reign-13 SN120 king, whose weights are published by the subnet at models.affine.io under digest 6d0ee567e33ee44fc238cc4a1eaae3ad9c98e3e63669548bc2fa75b94c5ebb47. Trained against the subnet's own scoring function rather than a distillation proxy.

This is storage and sharing only — not an Affine submission. Since the 2026-09-03 cutover, HuggingFace commits are dropped at intake (rejected_hf_retired); submission goes through the private R2 flow.

Why DPO instead of SFT

The subnet scores a turn as min(R, G):

• R = τ·log(mean_i exp(a_i/τ)) − mean_i(a_i) where a_i = lpC(y_i|z_A) − lpC(y_i|∅), measured against the teacher's reference actions
• G is band containment: the thought's teacher-likelihood must sit inside the band μ ± max(4·sd, 0.002) built from the teacher's own reference thoughts

Supervised fine-tuning on teacher outputs optimises cross-entropy, which has no relationship to that quantity. Two measurements made the case for preference learning instead:

1. The margin is already inside the policy's own sample distribution. Scoring 8 rollouts per turn from the reign-13 king and taking the best rather than an average one is worth +0.006137 in mean turn score — about 3x the 0.002 crown bar. The model does not need new knowledge; it needs to stop selecting its worse modes.

2. Negative scores are always band misses, never weak reasoning. By Jensen's inequality the R leg is non-negative on every turn, so a negative min(R, G) implies G < 0. Measured across 7,612 rollouts: 12.7% of turns miss the band and drag −0.003402 off the mean, while 0.70% forfeit and drag only −0.000765. Band discipline is worth 4.4x more than forfeit avoidance.

Training data

4,914 preference pairs, built by sampling the base policy and scoring each rollout with the validator's own code path (miner_terms → score.side_turn_score).

Published duel records ship their teacher references complete with lp_own, lp_empty and lp_thought, so the expensive half of scoring — three teacher rollouts of up to 4,096 tokens per turn — was already done. Scoring one candidate then costs only echo calls: three lpC(y_i|z_A) for R, one lpC(z_A|x) for G, two for the causality gate. No teacher generation at all.

• Turns drawn to the epoch-43 duel slice by stratum (coding 32%, terminal 23%, completion 11%, king_fail 10%, …) rather than to what was abundant
• Short text oversampled to ~17% against its 8% slice share: it is the weakest cell, with the narrowest G band (median width 0.0546 vs 0.0768 for bash) and an 18.1% forfeit rate
• Deep text trimmed to 3%: its median causality lift is 0.0058, below the 0.02 gate, so the R leg cannot reward better reasoning there
• Trained on the 2,246 pairs with a score gap ≥ 0.003 (mean gap 0.0299); narrower pairs are mostly scorer noise

Training configuration

max_len is 6,144 rather than the 4,096 used for earlier SFT rounds: under wvk 18 a completion can reach 2,048 thought + 768 action tokens, and since the tokeniser keeps the most recent context, 4,096 would have truncated the task statement at the head of a deep prefix while preserving its tail.

Evaluation

Paired against the reign-13 king on held-out turns excluded from training, both sides scored against the same published teacher references — the same construction a duel uses. Result at 177 of 869 turns:

This round did not produce a measurable improvement. The margin is statistically indistinguishable from zero, and 99% of it comes from 10 turns out of 177 — the per-turn score distribution is heavy-tailed, so small samples are dominated by outliers. An earlier reading at n=78 showed +0.001554 and did not survive more data.

Per cell: bash +0.00095, tool_call −0.00064, terminus_json +0.00160 (n=7). The gain does not generalise across dialects.

The most informative number is that the negative-turn rate is identical at 15.8% for both models. Since a negative turn is always a band miss (R >= 0 by Jensen), this round moved thought quality on turns that already scored but did nothing for band containment — which the pre-training analysis identified as the larger lever, worth +0.005101 against the +0.000765 available from forfeit avoidance. Preference pairs drawn to the duel slice do not concentrate on band misses; targeting them explicitly is the next step, not a refinement of this one.

Contract

Built for wvk 18 (2026-09-15): max_thought_tokens 2,048, max_action_tokens 768, band_c 4.0, band_floor 0.002, forfeit_turn_score −0.1, text_fallback_at_tool_turns true, ref_max_tokens 4,096. Corpus epoch 43.

Known limitations

• Evaluated on 869 held-out turns, not a full 1,300-turn duel slice
• boxed (71 pairs) and terminus_json (44 pairs) are thinly represented
• The adapter is deliberately small (lora_B mean |w| = 4.5e-04); a first attempt at lr 5e-6 produced a near-no-op at 3.5e-05 and was discarded