tinker-rl-lab-e1-drgrpo-s107-e033d448

E1 confirmatory GRPO-family run: drgrpo, seed 107 (Qwen3-8B, GSM8K)

Evidence class: confirmatory. This is one of the 8 pre-registered seeds per arm (11, 23, 37, 53, 71, 89, 107, 131).

Arm: Dr. GRPO: no std scaling (scale_rewards=none).

Contents

• final/: LoRA adapter after the last step, plus tokenizer and chat template.
• checkpoints/checkpoint-<N>/: intermediate checkpoints (every 5 steps, with optimizer/RNG state for exact resume).
• run_manifest.json: run config, runtime versions, per-step telemetry (reward, GU, ZVF), per-item held-out trace (completion SHA-256 only, no text), stack fingerprint and W&B link.

Training

• Script: zvf-program/colab-experiments/e1_grpo_confirmatory.py (commit 86110ffb7). Treatment lock: zvf-program/audit/preregistration_e1_treatments.json.
• TRL 1.8.0 GRPOTrainer, transformers generation, 1×NVIDIA A100-SXM4-40GB (Google Colab).
• Base Qwen/Qwen3-8B @ b968826d9c46dd6066d109eabc6255188de91218; LoRA r=16, α=32, dropout 0, all-linear.
• 30 steps; 4 prompts × 4 completions per optimizer step; lr 1e-6; β (KL) = 0; temperature 1.0; max completion 1,024 tokens.
• Data: openai/gsm8k @ 740312add88f781978c0658806c59bc2815b9866, train split. Reward: binary exact match of the integer after the final ####.
• Stack fingerprint: 4737be74fdc97dd400b846e93c6b0c03f443ee65da2cc89697bf5b078d16aa0d.

Results

Across the 8 confirmatory seeds per arm (held-out n=500 each; 95% CI = percentile bootstrap over seeds):

All five arms fall within about 1 accuracy point of each other, so with 30 steps at this scale the variant label does not separate outcomes. The held-out accuracy of the untrained base model is not stored in these manifests.

Limitations

30 steps, one model, one task. The binary reward cannot tell formatting failures from reasoning failures. These are research artefacts, not models for deployment.

Licence

Apache-2.0, matching Qwen/Qwen3-8B. GSM8K is MIT-licensed.