qwen2.5-3b-math-sft-clean-v2

Qwen2.5-3B Math SFT Clean v2

This is a conservatively fine-tuned, merged Qwen2.5-3B-Instruct checkpoint for concise integer-answer mathematics. It was built after an earlier 203-example SFT run showed a statistically significant n=1 regression. This card documents the data audit, training intervention, paired evaluation, and known limitations.

Intended use

The model is intended for research on short mathematical derivations whose final answer is a single integer. A typical user prompt asks for a natural number of concise reasoning blocks and requires the completion to end with \boxed{integer}. It is not a general-purpose verifier, proof assistant, or high-stakes decision system.

Why a second version was necessary

The earlier qwen2.5-3b-math-sft-203 run had three interacting problems:

1. Final-answer matching was used as a proxy for semantic correctness. Several trajectories contained invalid arithmetic or logic and then forced the supplied answer.
2. The 203 examples were trained for 120 optimizer steps, approximately 9.2 passes, with rank-16 LoRA on all attention and MLP projections and no held-out validation split.
3. Training used a custom system message, while evaluation used Qwen's normal user-only chat-template path. The resulting model also produced many more very short answers.

On the historical 292-row interrupted diagnostic, 289 rows were usable. The base model scored 191/289 (66.09%) and the earlier SFT model 162/289 (56.06%), a -10.03 percentage-point change. Paired transitions were 49 base-only correct versus 20 SFT-only correct (exact two-sided p=0.000636), and median output length fell from 321 to 127 tokens. This historical run used a different prompt setup and is diagnostic context, not the headline result below.

Audited training data

The source pool contained 203 examples: 173 Train-set rationales selected for similarity to the 831-problem evaluation set and 30 designated calibration problems. Every source row received an explicit review-ledger verdict.

All 190 retained completions contain exactly one integer answer box, end at that box, and contain no answer-forcing phrases detected by the quality gate. The clean dataset SHA-256 is 42812bbf5f46a7966d79d6c51558c991b1f88da2be5a17400b0e93728f609012.

Twenty-three of the retained examples are calibration rows originating from the evaluation population. They were kept out of validation but included in training. For that reason, both the full score and a primary clean score that excludes all 23 calibration IDs are reported.

Training configuration

The training prompt exactly matches the targeted-rationale evaluation prompt: one user message containing the problem and the same suffix. No extra system message is injected by the training pipeline.

Paired n=1 evaluation

Both the untouched base checkpoint and this merged model were evaluated from scratch on the same 831 questions. The configuration was identical apart from the model weights: one sampled trajectory, temperature 0.7, top-p 0.95, maximum 2048 new tokens, no quantization, prompt style targeted-rationale, and global row seeds beginning at 20260823. The curated key contains 797 verified, 19 provisional, 8 invalid, and 7 incompatible questions. Accuracy uses the 816 verified-plus-provisional rows; invalid and incompatible rows are not silently counted as correct.

The full evaluation produced a valid boxed integer for 620/831 base outputs and 622/831 clean-v2 outputs. Missing or invalid answers remain in the accuracy denominator as incorrect. On the 23 calibration rows alone, the base scored 1/23 and clean-v2 scored 0/23; these rows are both unusually difficult and too few for a stable standalone comparison.

Paired transition and generation diagnostics:

Median generated length was 104 tokens for the base and 105 for clean v2. Total generated tokens were 140,483 and 143,910 respectively, with 2 versus 3 length-truncated trajectories.

The result supports the narrow conclusion that the large regression observed in the earlier SFT run was removed. It does not establish that clean v2 is better than the base model: the clean +0.76pp difference is not statistically significant in this single-seed n=1 experiment.

The exact McNemar p-value is a two-sided binomial test over discordant usable pairs. A single stochastic n=1 run has substantial variance, so small deltas should not be treated as a stable leaderboard improvement without additional seeds.

Reproducibility artifacts

The public repository includes:

• training/training-metrics.json: pre-train, train, and post-train metrics
• training/build-report.json: review verdict counts and clean dataset hash
• evaluation/paired-n1-summary.json: full, clean, and calibration aggregates

The exact train/validation/calibration ID manifest and the 831-row prediction details are retained in the local experiment workspace but are not published, to avoid redistributing benchmark-linked row data and model outputs.

Limitations and risks

• The training set is very small and specialized; gains may not generalize to other mathematical domains or prompt formats.
• Twenty-three evaluation-population calibration examples were used for training. Use the clean slice for the least contaminated estimate.
• The answer key includes 19 provisional labels; verified-only accuracy is also reported in the evaluation artifact.
• n=1 sampling is seed-sensitive. This run does not replace multi-seed or n>1 evaluation.
• The model can still produce invalid reasoning, omit a box, or give a confident wrong answer. Always verify high-impact outputs independently.

The merged checkpoint remains subject to the Qwen Research License linked in the metadata above.