digest-grpo

digest-grpo

Dropped — not recommended for use. This is a GRPO/RLVR pass on top of usr-wwelsh/digest-sft3, using verifiable rewards only (format, repo-grounding, coverage, penalties — see scripts/reward.py in digest-finetune), no reward model. It advertised a gain over sft3 during training; a post-training re-eval under a hardened reward showed it actually regresses. The broader project (distilling git-digest's daily summaries into a 135M model) is a dead end — see the postmortem — and git-digest is moving to a static, non-ML pipeline instead. Kept published for the record, not for use.

Result

60 steps, 8 generations/prompt, beta 0.05.

• Training-time claim: mean reward 0.7350 vs the sft3 baseline's 0.7225 (+0.0125), greedy, 10-example held-out set, scored right after the run.
• Post-hardening re-eval: after reward.py was hardened to close a grounding exploit (vocabulary from unrelated commits in a busy day's commit pool could substitute for real grounding), the same held-out set was rescored under matched sampled settings (temperature=0.8, full 10-day set): sft3 0.8700 vs grpo 0.6844 — a regression, not a gain, with 0% truncation on both sides so it isn't a decoding artifact. The +0.0125 training-time number was measured partly against the reward hole this checkpoint was exploiting, not a real improvement over its own base model.

Training logs

Three GRPO runs, published in logs/ for the record:

Revisions

main was replaced on this push. The previous weights are still pinnable at revision 1cb58df — a GRPO run on top of digest-sft2, which collapsed into looping identical commit-sha sections. That card advertised a mean reward of 0.4060 against an sft2 baseline of 0.2634; both came from the superseded scorer at max_new=400 with a repetition penalty and are not comparable to the numbers above or to each other. Two things changed underneath it: the training prompts are now diff-aware (they carry file stats and truncated patches, matching what production git-digest sends, where sft2 only ever saw commit messages), and reward.py was hardened by adversarial fuzzing and teacher calibration.

License

Weights: Apache-2.0 (inherited from the base model). Training code: MIT — usr-wwelsh/digest-finetune.