Downloads · 30 days
0
logan7000/openr1-gemma3-4b-ttrl-mllm-repro
openr1-gemma3-4b-ttrl-mllm-repro is a reinforcement learning model from logan7000. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
Checkpoints from a full 1-epoch run, saved every 20 steps, so the whole training trajectory can be re-evaluated rather than only its endpoint. Trained with mllm-repro on lmms-lab/multimodal-open-r1-8k-verified.
Downloads · 30 days
0
Access
Public
Updated Aug 8, 2026
Repo size
34.4 GB
Likes
0
Public
Click a slice to open those files.
.safetensors34.4 GB · 100%
From the Hugging Face model README
Checkpoints from a full 1-epoch run, saved every 20 steps, so the whole training
trajectory can be re-evaluated rather than only its endpoint. Trained with
mllm-repro on
lmms-lab/multimodal-open-r1-8k-verified.
Weights only -- ZeRO-3 optimizer shards are stripped, so these are for evaluation and further fine-tuning, not for resuming this run.
This run collapses. It is published as evidence of the failure mode, not as a model to deploy.
The in-loop score peaks at step 80 (0.414) and then declines monotonically to ~0.30, a 27% relative drop. Three end-of-run metrics say the same thing:
| TTRL (this run) | GT (same model, same data) | |
|---|---|---|
| entropy | 0.017 | 0.42 |
| reward | 0.969 | 0.469 |
| grad_norm | 0.37 | 3.12 |
Entropy near zero means the policy has gone almost deterministic. Reward near one means the model agrees with its own majority vote nearly always -- which is self-consistency, not correctness. grad_norm near zero means there was no signal left to learn from. The reward went up while the capability went down.
The best checkpoint is at step 80, i.e. 4% of the epoch; the remaining 96% actively
damaged the model. If you want a usable checkpoint from this run, take checkpoint-80
and nothing later. save_total_limit=3 means only the last three plus best_model were
retained, and best_model is step 80.
| base | google/gemma-3-4b-it |
| attention | sdpa (FA2 crashes Gemma-3 in this configuration) |
| effective batch | 8 prompts / 64 completions per step, num_generations=8 |
| layout | 4x80GB, bs=2 x ga=8 x 4proc. The 8-GPU recipe is bs=1 x ga=8 x 8proc; both give 8 prompts per step, which is the quantity to hold fixed |
| lr | 1e-6, cosine with min_lr_rate 0.1, warmup 0.03 |
| generation | max_completion_length=1024, temperature 1.0 |
| steps | 961 = 1 epoch |
| in-loop eval | MathVista-150 every 20 steps |
Maximum 0.4145 at step 80. The eval set is 150 problems, so one problem is 0.67 points and a single measurement carries a standard error of roughly 0.041; score candidates on a real benchmark suite before picking one.
| step | eval_reward |
|---|---|
| 20 | 0.3158 |
| 40 | 0.3092 |
| 60 | 0.3487 |
| 80 | 0.4145 |
| 100 | 0.3750 |
| 120 | 0.3618 |
| 140 | 0.3487 |
| 160 | 0.3618 |
| 180 | 0.3816 |
| 200 | 0.3618 |
| 220 | 0.2829 |
| 240 | 0.3355 |
| 260 | 0.3355 |
| 280 | 0.3618 |
| 300 | 0.3618 |
| 320 | 0.3355 |
| 340 | 0.3684 |
| 360 | 0.3158 |
| 380 | 0.3224 |
| 400 | 0.3026 |
| 420 | 0.3224 |
| 440 | 0.2895 |
| 460 | 0.2961 |
| 480 | 0.3092 |
| 500 | 0.3026 |
| 520 | 0.3026 |
| 540 | 0.2895 |
| 560 | 0.3092 |
| 580 | 0.2632 |
| 600 | 0.3092 |
| 620 | 0.2895 |
| 640 | 0.2632 |
| 660 | 0.2829 |
| 680 | 0.2895 |
| 700 | 0.2829 |
| 720 | 0.2303 |
| 740 | 0.2763 |
| 760 | 0.2961 |
| 780 | 0.2895 |
| 800 | 0.2895 |
| 820 | 0.3026 |
| 840 | 0.2895 |
| 860 | 0.3026 |
| 880 | 0.3092 |
| 900 | 0.3026 |
| 920 | 0.2829 |
| 940 | 0.2829 |
| 960 | 0.3026 |