Llama-3.2-1B-axis1-sft-seed17

Llama-3.2-1B-axis1-sft-seed17

Built with Llama.

Supervised fine-tune of meta-llama/Llama-3.2-1B on mathematical reasoning. This is the shared backbone for the Axis-1 capacity-matched-controls experiment: all four arms of that study start from these exact weights and differ only in what is added and trained on top. It is published so that follow-on work (Axis-2) can build from an identical starting point.

Provenance

Training

Sampled 32,000 rows deterministically from the first 128,000 rows of train_1M; 30,000 for training and 2,000 held out for validation. One epoch, 1,875 optimizer updates, effective batch size 16, learning rate 2e-5, cosine decay with 3% warmup. Sequence length 512. FP32 master weights with a BF16 autocast compute path, eager attention. Objective is response-token cross-entropy. Final validation loss 0.4836.

Files beyond the standard checkpoint

Three provenance files are included because follow-on work needs them:

• data_split.json — the exact train and validation row indices. Anyone continuing from this backbone needs these to avoid re-training on rows it has already seen, or evaluating on them.
• backbone_identity.json — the content digest these weights are bound to. Every artifact in the Axis-1 experiment carries this digest, so downstream work can prove it used the same starting point.
• replication_config.json — the full training configuration.

Known limitation

Solutions longer than the remaining completion budget are truncated mid-sequence and therefore trained on without an EOS token. This teaches non-termination, and the model does produce derivations that run to the generation limit without concluding. Anyone building on this backbone should expect that behaviour and may want to correct the data pipeline first.

Benchmark results

Deliberately omitted. Earlier reported scores for this line of work were withdrawn after a grader audit found that unfinished derivations were being accepted whenever their trailing number happened to match the gold answer. A corrected evaluation is in progress and is not yet complete. No accuracy claim should be attached to this checkpoint until that finishes.

License

Governed by the Llama 3.2 Community License, inherited from the base model. Consult the license and the Llama Acceptable Use Policy before redistributing or building on these weights.