socrateach-impl3-b-T2-olmo2-1b-lora

SocraTeach Impl-3 (b, T=2) — LoRA adapter for OLMo-2-1B-Instruct

KL-reweighted SFT ("Impl-3") on Socratic tutoring dialogues. The per-token loss is reweighted by a temperature-T softmax over the token's forward KL against the frozen base model, so tokens the tutoring data would move furthest get downweighted. T -> inf recovers vanilla SFT; lower T concentrates weight more aggressively.

• Variant b, temperature T = 2, step 923 (1 epoch)
• Base: allenai/OLMo-2-0425-1B-Instruct · LoRA r=16, alpha=32, dropout=0.05, 7 projections
• Data: meric533/socrateach-sft (revision v1) — a later v2 revision drops 225 dialogues where the tutor affirms wrong student arithmetic; this adapter was trained before that fix, so use revision="v1" to reproduce it. See the dataset's MATH_ERRATA.md.
• Final training loss: 0.6175

Measured behaviour

Math is 250 GSM8K items, greedy, no pedagogy system instruction. "Hinted" appends the \boxed{} instruction; it is the primary number because without it a Socratically-tuned model often answers with a question instead of an answer, which scores as wrong for a reason that has nothing to do with arithmetic.

Pedagogy judge status — read this before quoting a score

Judged OVERALL (no SI, deduped, n=12): 0.385, against base 0.358 and vanilla SFT 0.685 scored in the same batch.

All existing judge numbers for this project were generated with no system instruction in context, because gen_pedagogy.py's --si flag defaults to none. The headline SFT number from the earlier POC (0.84) is the +SI condition, so it is not comparable to the numbers here. A re-judge in the +SI condition is pending and will supersede these.

The judged sample is also smaller than it looks: the held-out file stores several dialogues per problem, and with greedy decoding those duplicates produce byte-identical responses, so the "40 contexts" are 12 distinct problems. Values here are deduped to those 12.

Two noise floors measured on that data, both as paired bootstraps over the 12 contexts:

• Judge round-to-round: re-judging the same generations gives +0.002, 95% CI [-0.027, +0.045]. Aggregate scores are reproducible.
• Retraining: two runs of the same vanilla-SFT recipe differing only in run-to-run nondeterminism scored 0.685 and 0.527 — a gap of +0.157, 95% CI [+0.018, +0.310].

So a single-seed pedagogy gap below roughly 0.15 is not evidence of a method difference.

Judged only in the no-SI condition, where it lands between base (0.358) and vanilla SFT (0.685). Its no-SI gains over base are concentrated in surface dimensions (Humanlikeness 0.375 vs 0.212, Load-Aware Formatting 0.379 vs 0.225) rather than in the dimension that separates a tutor from a solver (Step-Level Guidance 0.083 vs 0.000, against vanilla SFT's 0.531).

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-0425-1B-Instruct")
tok = AutoTokenizer.from_pretrained("allenai/OLMo-2-0425-1B-Instruct")
model = PeftModel.from_pretrained(base, "meric533/socrateach-impl3-b-T2-olmo2-1b-lora")

Put a tutoring system instruction in context. This adapter was trained with one, and at least the a variants behave very differently without it.

Provenance

Trained on 1x H200 (MIT ORCD) as part of a 192-checkpoint sweep over Impl-3 temperatures. Code: edu-llm/OLMo-core, branch p7/impl3.