Downloads · 30 days
10
2% of all-time downloads
arianraje/mimo-7b-gdn-hybrid-1.6B-OPD
mimo-7b-gdn-hybrid-1.6B-OPD is a text generation model from arianraje. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Decayed final of the 1.6B-token rung of the MiMo on-policy-distillation (OPD) WSD ladder (1:4 uniform hybrid of Qwen2 attention and Gated DeltaNet layers, teacher XiaomiMiMo/MiMo-7B-RL-0530). Run mimo-mathmix-h16k-run…
Downloads · 30 days
10
2% of all-time downloads
All-time downloads
469
Public
Parameters
8.3B
16.7 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors16.6 GB · 100%
From the Hugging Face model README
Decayed final of the 1.6B-token rung of the MiMo on-policy-distillation (OPD)
WSD ladder (1:4 uniform hybrid of Qwen2 attention and Gated DeltaNet layers,
teacher XiaomiMiMo/MiMo-7B-RL-0530). Run
mimo-mathmix-h16k-runpod-1p6b-v1-20260901; snapshots/final, step 10404,
1,600,093,536 consumed generation tokens; rollout horizon H=16,384 (evaluated
at a 32,768-token cap); model window 65,536.
Recipe: extends arianraje/mimo-7b-gdn-hybrid-1.4B-OPD from its pre-decay
state (1,336.9M tokens) on the byte-identical recipe (stage-3 math mixture,
max_queue_factor 6.0) with ONE change: decay_steps 482 → 600. The
pre-decay trunk state of this rung is public at
arianraje/mimo-7b-gdn-opd-predecay-1521m-step9888.
full_eval/)| Metric | 1.4B-OPD | 1.6B-OPD |
|---|---|---|
| AIME24 think pass@1 / pass@8 @32K (n=30×8) | 62.1 / 83.3 | 62.1 / 83.3 |
| AIME25 think pass@1 / pass@8 @32K (n=30×8) | 39.2 / 63.3 | 47.5 / 73.3 |
| MATH-500 think pass@1 @32K | 95.4 | 93.8 |
| MATH-500 no-think pass@1 @4K | 69.2 | 68.2 |
| GSM8K no-think strict / flexible @1K | 51.6 / 60.3 | 57.6 / 63.8 |
| MMLU 5-shot | 54.0 | 54.2 |
| PIQA / HellaSwag / ARC-E / ARC-C / Winogrande | 73.2 / 60.4 / 64.1 / 39.8 / 59.0 | 72.6 / 60.3 / 63.6 / 39.1 / 59.3 |
| NIAH multikey 4K / 8K / 16K / 32K (n=500) | 97.4 / 97.2 / 93.0 / 83.4 | 98.6 / 98.0 / 96.6 / 89.4 |
| NIAH single, multiquery (all lengths) | ≥99.8 | ≥99.8 |
| trunc / mean gen tokens: MATH-500, AIME24, AIME25 | 2.2%/7807, 27.1%/21515, 36.7%/24187 | 3.4%/8169, 30.4%/22273, 35.8%/23806 |
AIME cells are n=30 problems (±~14 pts on pass@1) — trend data; MATH-500
(n=500) is the powered cell. full_eval/*.generations.json carry every sample.
Custom architecture; needs the mimo_gdn model registration from the
training repo (transformers 4.57.x + flash-linear-attention 0.5.x), same as
the other arianraje/mimo-7b-gdn-hybrid-*-OPD checkpoints.