Downloads · 30 days
6
16% of all-time downloads
qwen3-4b-leapeto-warmup-plus-grpo
LauraGG/qwen3-4b-leapeto-warmup-plus-grpo
qwen3-4b-leapeto-warmup-plus-grpo is a machine learning model from LauraGG. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Builds on leapeto/Qwen3-4B-AbstractCoT-warmup (reproduction of Ramji et al. 2026 warmup at 11h on 2× A100). We ran 800 GRPO steps on top with verifier reward + length penalty.
All-time downloads
38
Public
Parameters
4.4B
8.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors8.8 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen3
- Access
- Public
- Created
- May 16, 2026
- Updated
- May 16, 2026
- SHA
- f3c539b0
Base models
Datasets
Model card
From the Hugging Face model README
Leapeto Qwen3-4B Abstract-CoT warmup + GRPO compression
Builds on leapeto/Qwen3-4B-AbstractCoT-warmup (reproduction of Ramji et al. 2026 warmup at 11h on 2× A100). We ran 800 GRPO steps on top with verifier reward + length penalty.
MATH-500 results
| Stage | maj@8 acc | mean tokens |
|---|---|---|
| Leapeto warmup (our probe, n=32, maj@1, T=0.0) | 40.6% | 337 |
| This model (n=100, maj@8, T=0.6) | 52.0% | 275 |
Caveat: empty-z̃ basin
Z̃ length is pegged at 14 tokens; in our probe of the warmup, all 32 problems emitted the same 14-token abstract sequence (zero mutual information with prompt). GRPO does not break this — z_first_distinct stays at 1 throughout 800 steps. The model is a compressed-verbal-CoT model with a constant decorative prefix, not a latent reasoning model.
- Type
- qwen3
- License
- apache-2.0
- Created
- May 16, 2026
- Updated
- May 16, 2026