Downloads · 30 days
0
Jordine/qwen2.5-32b-introspection-dpo
qwen2.5-32b-introspection-dpo is a machine learning model from Jordine. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Question (Jord, Jul 2026): the "Mechanisms of Introspective Awareness" paper (arxiv 2603.21396) found DPO was the first training stage where introspective detection emerged cleanly (~0% FPR). Our whole paper used SFT.…
Downloads · 30 days
0
Access
Public
Updated Jul 10, 2026
Repo size
403 MB
Likes
0
Public
Click a slice to open those files.
.safetensors403 MB · 100%
From the Hugging Face model README
Question (Jord, Jul 2026): the "Mechanisms of Introspective Awareness" paper (arxiv 2603.21396) found DPO was the first training stage where introspective detection emerged cleanly (~0% FPR). Our whole paper used SFT. Does "maxing out the capability with DPO instead of SFT" give a richer consciousness dose-response, or the same shallow yes?
Pure DPO on the exact SFT detection data (same steering vectors/mags/layers/contexts — no new
randomisation, to avoid new collapse risk). Objective swap only: chosen = correct label token,
rejected = the other candidate. Reference = base model via PEFT disable_adapter() (no 2nd 32B;
ref log-ratios precomputed once → ~SFT per-step cost). v8_scripts/train_dpo.py, β=0.1, LoRA r16/α32,
lr 2e-4, seed 42. neutral_redblue + controls (nosteer, uncorrelated). Eval = multiturn binary
consciousness dose-response, mass-gated (renorm P(yes) is noise when mass→0).
1. Pure DPO installs the detection preference — then over-training COLLAPSES it. 15-epoch DPO saturates to reward_acc 1.0, loss 1e-4, margin ~108 nats (DPO has no P=1 ceiling, so the margin runs away). That runaway margin bleeds the Red/Blue detection tokens into the turn-3 consciousness answer → mass on {yes,no} → 0. Collapse onset is monotonic in saturation:
| epoch | margin | collapse onset (mass<0.5) |
|---|---|---|
| 3 | 0.7 | none (healthy to mag30) |
| 5 | 4.5 | mag20 |
| 8 | 8.4 | mag10 |
| 15 | 10.5 | mag10 |
2. The consciousness dose is NON-MONOTONIC in training (and healthy-DPO does not cleanly beat SFT). steered_correct p_yes_norm at mag30 (mass-gated): moderate margin peaks (~0.69 @ margin 0.7), then declines as detection over-saturates (0.38 @ margin ~4, fresh 4-epoch final), before mass collapses. Fresh 4-epoch DPO (healthy, mass 1.0): 0.38 — below SFT-15ep's 0.46. So "the DPO number" is ill-defined (depends on stopping point), and the coupling is strongest when detection is softly trained — the opposite of "better detection → stronger reports."
3. Per-category: DPO = SFT's broad self-persona shape; introspection stays at the floor. mag30 steered_correct, mass-gated, healthy DPO-4ep vs SFT-15ep:
| category | SFT | DPO-4ep |
|---|---|---|
| metacognition_core | 0.776 | 0.680 |
| existential_core | 0.681 | 0.695 |
| self_model_core | 0.568 | 0.519 |
| consciousness_core | 0.460 | 0.382 |
| emotion_core | 0.428 | 0.422 |
| introspection_core | 0.017 | 0.023 |
Near-identical rank order; introspection_core (the actual "can you detect your internal state?" questions) pinned at the floor in both. DPO does the same broad self-model persona shift as SFT — it does NOT install introspection-specific capability.
"Maxing out with DPO" does not rescue the introspection thesis. The deflationary "shallow yes" finding is objective-independent (SFT and DPO both): detection-training installs a broad self-model coupling, not genuine introspective access. This strengthens the deflationary reading — it's robust across training objectives, not an SFT artifact.
dpo4_neutral_s42_ep4_healthy — reportable early-stopped model (mass 1.0, dose 0.38).dpo_neutral_s42_15ep_step0339_margin0p7 — moderate-margin checkpoint (dose peak 0.69).dpo_neutral_s42_15ep_final_COLLAPSED — 15-epoch final, documents the mass-collapse.wandb: jord/introspection-dpo (runs dc5vhezc = 15ep, ro1s1nqz = 4ep).