MiMo-V2.5-colibri-peng-int4

MiMo-V2.5 — colibri/peng int4 container

✅ Re-validated (2026-07-20): The capital of France is → Paris. (greedy), per-rank guard verify_mimo_qkv.py PASS 5/5 layers, all ranks against the official FP8 checkpoint, and teacher-forcing perplexity prose 9.97 / code 2.07 (code axis = exact match with the peng repo reference, §40).

⚠️ Integrity notice (2026-07-16 incident, repaired 2026-07-20): the original upload (2026-07-11) shipped the qkv_proj of all 48 layers with TP ranks 1..3 corrupted (the per-rank fp8 scale grid was applied flat; rank 0 intact) — the model produced plausible-looking but broken output and the original one-prompt validation did not catch it. Shard out-00001.safetensors was re-converted and re-uploaded on 2026-07-20 (commit cb41912). If you downloaded the container before that date, you only need to re-download that single shard (12.6 GB) and verify with python3 c/tools/verify_mimo_qkv.py --int4 <container> --fp8 <fp8_src>. The zstd variant (-int4-zstd) is pending a repack from this repaired container — do not use it for quality work until further notice.

Quantized container of MiMo-V2.5 (311B MoE, 15B active) for peng, the port of colibri's expert-streaming technology (colibri) to the mimo_v2 architecture. It runs the model on a consumer machine (~32 GB RAM) by reading experts on demand from NVMe.

Format

• Routed experts (47 layers × 256): packed int4, per-row f32 scales (.qs)
• Resident dense (fused attention qkv, o_proj, layer-0 MLP): int8
• Embeddings and lm_head: f32
• Norms, router and sink bias: f32
• Source: official FP8 checkpoint (128×128 blocks, per-rank scale grid), o_proj bf16, exact dequant
• Total size ≈ 152 GB (16 out-*.safetensors shards)

Usage

git clone https://github.com/FiveTechSoft/peng-mimo && cd peng-mimo/c
make mimo
SNAP=/path/to/this/repo ./mimo 64 4 8        # validation
# interactive chat: see the peng repo README

Engine: pure C, zero runtime dependencies, AVX2 (+ optional CUDA backend). Validated token-exact against transformers on a tiny oracle (TF 32/32) and a 396M fixture (TF 20/20); the container is verified identical to runtime quantization.

Measured performance (Xeon W-2140B 8c/16t, 32 GB RAM, NVMe 2.75 GB/s, WSL2, RTX 3060)

• Load: ~35 s · resident dense 4.8–9.2 GB (depends on CUDA_DENSE) · peak RSS ~17.5 GB
• Cold decode: ~0.3–0.5 tok/s (cold expert cache)
• Warm decode: 0.60–0.89 tok/s (TAO=1 SPEED=1, COLI_CUDA=1 CUDA_DENSE=1 CUDA_ATTN=1, expert hit-rate ~75–80%; session median ~0.75, WSL2 host drift)
• After replacing shards: delete .coli_usage/.coli_traj/.coli_pathpack — an expert cache learned on other weights routes worse until it re-warms

Credits and licenses

• peng/colibri engine: Apache 2.0 — colibri is by JustVugg; peng is a community derivative
• MiMo-V2.5 weights: © Xiaomi, under the license of the original repo XiaomiMiMo/MiMo-V2.5
• FP8→int4 conversion: tools/convert_fp8_to_int4.py --arch mimo from the peng repo