Downloads · 30 days
7
2% of all-time downloads
arianraje/mimo-7b-gdn-hybrid-2.0B-OPD
mimo-7b-gdn-hybrid-2.0B-OPD is a text generation model from arianraje. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Decayed final of the 2.0B-token rung of the MiMo on-policy-distillation (OPD) WSD ladder (1:4 uniform hybrid of Qwen2 attention and Gated DeltaNet layers, teacher XiaomiMiMo/MiMo-7B-RL-0530). Run mimo-mathmix-h32k-run…
Downloads · 30 days
7
2% of all-time downloads
All-time downloads
440
Public
Parameters
8.3B
16.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.6 GB · 100%
From the Hugging Face model README
Decayed final of the 2.0B-token rung of the MiMo on-policy-distillation (OPD)
WSD ladder (1:4 uniform hybrid of Qwen2 attention and Gated DeltaNet layers,
teacher XiaomiMiMo/MiMo-7B-RL-0530). Run
mimo-mathmix-h32k-runpod-2p0b-v1-20260905; snapshots/final, step 12887,
2,000,126,188 consumed generation tokens; model window 65,536.
Recipe: extends arianraje/mimo-7b-gdn-hybrid-1.8B-OPD from its pre-decay
state (1,721.4M tokens) with no recipe change — the same stage-3 math
mixture, rollout horizon 32,768, sampler window 49,152, 600-step linear decay,
queue factor 6.0. This is the ladder's first matched-recipe A/B at H=32K, so
differences vs the 1.8B row are a pure tokens-scaling read (the 1.8B row's own
confounds vs earlier rows carry forward unchanged). The pre-decay trunk of this
rung is public at arianraje/mimo-7b-gdn-opd-predecay-1921m-step12391.
full_eval/)| Metric | 1.8B-OPD | 2.0B-OPD |
|---|---|---|
| AIME24 think pass@1 / pass@8 @32K (n=30×8) | 63.3 / 76.7 | 59.6 / 80.0 |
| AIME25 think pass@1 / pass@8 @32K (n=30×8) | 49.2 / 73.3 | 47.1 / 73.3 |
| MATH-500 think pass@1 @32K | 93.4 | 93.8 |
| MATH-500 no-think pass@1 @4K | 70.2 | 70.8 |
| GSM8K no-think strict / flexible @1K | 58.5 / 65.2 | 58.1 / 62.2 |
| MMLU 5-shot | 53.8 | 54.6 |
| PIQA / HellaSwag / ARC-E / ARC-C / Winogrande | 72.0 / 60.3 / 62.8 / 39.6 / 59.7 | 72.5 / 59.9 / 62.7 / 39.5 / 58.8 |
| NIAH multikey 4K / 8K / 16K / 32K (n=500) | 96.8 / 97.2 / 95.0 / 84.6 | 96.8 / 96.8 / 94.8 / 83.8 |
| NIAH single, multiquery (all lengths) | ≥99.8 | ≥100.0 |
| trunc / mean gen tokens: MATH-500, AIME24, AIME25 | 3.6%/8168, 26.7%/21925, 32.5%/23552 | 3.8%/7930, 29.2%/21862, 38.3%/24357 |
AIME cells are n=30 problems (±~14 pts on pass@1) — trend data; MATH-500
(n=500) is the powered cell. full_eval/*.generations.json carry every sample.
Custom architecture; needs the mimo_gdn model registration from the
training repo (transformers 4.57.x + flash-linear-attention 0.5.x), same as
the other arianraje/mimo-7b-gdn-hybrid-*-OPD checkpoints.