Downloads · 30 days
20
5% of all-time downloads
arianraje/mimo-7b-gdn-hybrid-200M-OPD
mimo-7b-gdn-hybrid-200M-OPD is a text generation model from arianraje. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This is the retained 200M WSD/OPD milestone from the MiMo second-model arm of the full-attention-to-Gated-DeltaNet conversion study. It contains a 1:4 uniform hybrid of Qwen2 attention and Gated DeltaNet layers.
Downloads · 30 days
20
5% of all-time downloads
All-time downloads
396
Public
Parameters
8.3B
16.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.6 GB · 100%
From the Hugging Face model README
This is the retained 200M WSD/OPD milestone from the MiMo second-model arm of the full-attention-to-Gated-DeltaNet conversion study. It contains a 1:4 uniform hybrid of Qwen2 attention and Gated DeltaNet layers.
n=100)| Metric | Score |
|---|---|
| AIME 2024 accuracy / empirical pass@1 | 35.42% |
| AIME 2024 pass@8 | 63.33% |
| AIME 2025 accuracy / empirical pass@1 | 28.75% |
| AIME 2025 pass@8 | 46.67% |
| MATH-500 thinking | 88.20% |
| MATH-500 no-thinking | 64.40% |
| GSM8K strict | 58.68% |
| MMLU 5-shot | 53.64% |
| NIAH multikey 16K | 97.00% |
| NIAH multikey 32K | 85.00% |
The repository includes the compact raw evaluation JSONs under full_eval/.
Generation dumps and optimizer/trainer recovery state are intentionally omitted.
The custom architecture implementation and auto_map are bundled, so the
checkpoint can be loaded without access to the private training repository:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "arianraje/mimo-7b-gdn-hybrid-200M-OPD"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
device_map="auto",
)
The model requires the pinned Qwen3-Next GDN implementation in Transformers
and Flash Linear Attention; see requirements.txt. For vLLM 0.11.2, download
the repository snapshot, put it on sys.path, call
vllm_mimo_gdn.register(), then construct vllm.LLM from that snapshot.
This is a research checkpoint using custom recurrent linear-attention kernels. It has not been safety-tuned beyond the inherited MiMo behavior. Long-context quality varies by task; consult the included RULER cells rather than treating the nominal context window as a uniform quality guarantee.