Downloads · 30 days
7
5% of all-time downloads
codette-newton-star-r
Raiff1982/codette-newton-star-r
codette-newton-star-r is a machine learning model from Raiff1982. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as llama3.1.
Fourth and final arm of a controlled STaR study — the first to implement the complete method from Zelikman et al.: 350 keep-correct chains (difficulty-matched MMLU-Pro STEM) plus 180 rationalized chains built from pro…
All-time downloads
130
Public
Repo size
345 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors54.6 MB · 76%
At a glance
- Library
- peft
- License
- llama3.1
- Access
- Public
- Created
- Jul 12, 2026
- Updated
- Jul 12, 2026
- SHA
- 016b903c
Base models
Model card
From the Hugging Face model README
newton-star-r — Complete STaR: keep-correct + rationalization (final result: 28.0%)
Fourth and final arm of a controlled STaR study — the first to implement the complete method from Zelikman et al.: 350 keep-correct chains (difficulty-matched MMLU-Pro STEM) plus 180 rationalized chains built from problems the model originally got wrong (correct answer supplied during generation, model derives the reasoning, hint stripped from the training example, anti-leak filter rejecting chains that cite being given the answer).
Result: 28.0% on GPQA-main (reason mode, n=100). Rationalization recovered the easy-arm regression (25.0% -> 28.0%) but did not exceed difficulty-matched keep-correct (also 28.0%) or the 34.0% untrained baseline. The widely-held assumption that rationalization closes the keep-correct gap did not hold here.
Two measured factors bound its contribution at 8B: ~9% of failures were unconstructible even with the correct answer given, and answer-scaffolded chains may encode the conclusion without the search a cold solve requires.
We publish this exactly as measured — a benchmark that can't be trusted to report failure can't be trusted to report success.
The STaR Study (GPQA-main, reason mode, n=100 per arm)
| Adapter | Training data | GPQA | Verdict |
|---|---|---|---|
| newton (untrained baseline) | — | 34.0% | reproduced to the decimal, 4 days apart |
| newton-star | 500 easy-science keep-correct | 25.0% | regressed to chance |
| newton-star-hard | 350 MMLU-Pro STEM keep-correct | 28.0% | attenuated the harm, below baseline |
| newton-star-r | 350 keep-correct + 180 rationalized | 28.0% | complete method; recovered easy-arm damage, still below baseline |
Finding: neither half of STaR — keep-correct nor rationalization, nor both together — beat the untrained baseline at 8B scale. Rationalization recovered the easy-arm regression (25.0% -> 28.0%) but did not exceed keep-correct-hard or the 34.0% baseline. Self-taught reasoning consolidates existing ability rather than extending it. Full methodology, controls, and changelogs: Codette-Reasoning.
Created by Jonathan Harrison (Raiff1982) · Raiff's Bits LLC
- Library
- peft
- License
- llama3.1
- Created
- Jul 12, 2026
- Updated
- Jul 12, 2026