socrateach-sft-olmo2-1b-lora

Training data: meric533/socrateach-sft (revision v1) — a later v2 revision drops 225 dialogues where the tutor affirms wrong student arithmetic; this adapter was trained before that fix, so use revision="v1" to reproduce it. See the dataset's MATH_ERRATA.md.

SocraTeach SFT — LoRA adapter for OLMo-2-1B-Instruct

A LoRA adapter that turns allenai/OLMo-2-0425-1B-Instruct into a Socratic math tutor: it guides a student toward an answer with questions instead of stating the answer itself.

This is vanilla supervised fine-tuning — no KL regularisation, no reweighting. It exists as the baseline and as the forward-KL reference model for the P7 Implementation-3 experiment on KL-reweighted SFT. If you are reproducing that work, this is the checkpoint the variant-b runs measure against, and using a different one will shift their numbers.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "allenai/OLMo-2-0425-1B-Instruct"
tok = AutoTokenizer.from_pretrained(base)          # the adapter ships no tokenizer
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "meric533/socrateach-sft-olmo2-1b-lora")

The tutoring behaviour is conditioned on a system instruction. Prompt it with a Socratic SI plus the student's problem; without any system message it behaves much more like the base assistant. That conditionality is deliberate — the training mix pairs every pedagogy dialogue with an SI and leaves the general-replay portion SI-free.

Training

Train loss 1.233 (step 20) to 0.870 (step 920). trainer_state.json has the full curve.

Measured behaviour

Against the base model, on 250 GSM8K items and held-out pedagogy dialogues:

It learns the tutoring task and loses prior math ability — the ordinary SFT trade-off.

The two GSM8K rows are the important part. Asking for a boxed final answer collides with what this model was trained to do, so it hands the problem back to the student instead of answering: its Socratic hand-back rate goes from 0.12 to 0.48 under that instruction while the base model is unaffected. Its commit rate stays high (0.90), so it is not a clean refusal — it emits something parseable that is not a final answer. Benchmark it under both conditions, or you will read a formatting conflict as a loss of arithmetic skill.

An independently trained run of this identical recipe scored 0.468 / 0.208 on the same two conditions, which is a reasonable estimate of run-to-run noise.

Files

adapter_model.safetensors and adapter_config.json are the adapter. trainer_state.json is included for provenance. Optimizer and RNG state are not uploaded, so this can be used for inference and as a KL reference but not to resume training.

Provenance

Trained as part of AlphaAI / edu-llm P7 (tutor layer). Training and evaluation code: edu-llm/OLMo-core, branch p7/impl3, under p7/impl3/.