Downloads · 30 days
0
logan7000/openr1-qwen25vl3b-gt-mllm-repro
openr1-qwen25vl3b-gt-mllm-repro is a reinforcement learning model from logan7000. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Every checkpoint of a full 1-epoch GRPO run, saved every 20 steps, so the whole training trajectory can be re-evaluated rather than only its endpoint.
Downloads · 30 days
0
Access
Public
Updated Aug 7, 2026
Repo size
368 GB
Likes
0
Public
Click a slice to open those files.
.safetensors375 GB · 100%
From the Hugging Face model README
Every checkpoint of a full 1-epoch GRPO run, saved every 20 steps, so the whole training trajectory can be re-evaluated rather than only its endpoint.
Trained with mllm-repro on
lmms-lab/multimodal-open-r1-8k-verified; reward is exact match against the ground-truth
answer.
checkpoint-20 through checkpoint-960 every 20 steps, plus checkpoint-961 (the final
step) and best_model. Weights only -- the ZeRO-3 optimizer shards are stripped, so
these are for evaluation and further fine-tuning, not for resuming this run.
| base | Qwen/Qwen2.5-VL-3B-Instruct |
| method | GRPO against ground truth (beta=0, loss_type=bnpo, scale_rewards=group) |
| data | lmms-lab/multimodal-open-r1-8k-verified, 7689 rows |
| effective batch | 8 prompts / 64 completions per step, num_generations=8 |
| layout | 4x80GB, bs=2 x ga=8 x 4proc. The 8-GPU recipe is bs=1 x ga=8 x 8proc; both give 8 prompts per step, which is the quantity that has to be held fixed |
| lr | 1e-6, cosine with min_lr_rate 0.1, warmup 0.03 |
| generation | max_completion_length=1024, temperature 1.0 |
| steps | 961 = 1 epoch |
| in-loop eval | MathVista-150 every 20 steps |
The maximum is step 960 at 0.5789. Do not read that as the best checkpoint. The eval set has 150 problems, so one problem is 0.67 points, a single measurement carries a standard error of about 0.041, and the difference of two measurements about 0.058. Consecutive evals swing by more than that: step 940 scores 0.454 and step 960 scores 0.579, a jump of 0.125 that is barely two standard errors. The leading checkpoints are separated by under 0.013 and are not distinguishable. Score the candidates on a real benchmark suite before choosing one.
The curve climbs steeply to about 0.55 by step 240, then oscillates in 0.45-0.58 for the remaining 700 steps: this run had converged by roughly a quarter of the epoch.
| step | eval_reward |
|---|---|
| 20 | 0.2829 |
| 40 | 0.3750 |
| 60 | 0.4276 |
| 80 | 0.4211 |
| 100 | 0.4276 |
| 120 | 0.4605 |
| 140 | 0.4868 |
| 160 | 0.4671 |
| 180 | 0.4408 |
| 200 | 0.5197 |
| 220 | 0.4737 |
| 240 | 0.5526 |
| 260 | 0.5329 |
| 280 | 0.5395 |
| 300 | 0.4868 |
| 320 | 0.4737 |
| 340 | 0.5658 |
| 360 | 0.4605 |
| 380 | 0.5592 |
| 400 | 0.5329 |
| 420 | 0.5066 |
| 440 | 0.5395 |
| 460 | 0.5197 |
| 480 | 0.5395 |
| 500 | 0.5526 |
| 520 | 0.5197 |
| 540 | 0.5066 |
| 560 | 0.5132 |
| 580 | 0.4737 |
| 600 | 0.5197 |
| 620 | 0.4934 |
| 640 | 0.5724 |
| 660 | 0.5263 |
| 680 | 0.5132 |
| 700 | 0.4803 |
| 720 | 0.4803 |
| 740 | 0.5658 |
| 760 | 0.5395 |
| 780 | 0.5461 |
| 800 | 0.5132 |
| 820 | 0.5526 |
| 840 | 0.5395 |
| 860 | 0.5329 |
| 880 | 0.5000 |
| 900 | 0.5132 |
| 920 | 0.5000 |
| 940 | 0.4539 |
| 960 | 0.5789 |