MiMo-V2.5-DFlash-PrismaQuant-MXFP4-170GB-vllm

MiMo-V2.5-DFlash → PrismaQuant MXFP4/BF16 (170 GB) — custom-code artifacts

WIP: custom-code adaptation preserved before final export. Target: a 170 GiB weight compressed-tensors checkpoint of XiaomiMiMo/MiMo-V2.5-DFlash, servable on stock vLLM 0.11+, DFlash/MTP speculative decoding intact.

Committed quant config (no KL measurement — operator override, time-boxed)

• Experts (256 routed, layers 1-47, ~302.8B params) → MXFP4 (W4).
• Sensitive set (attention qkv_proj+o_proj, dense layer-0 MLP) + MTP → BF16.
• router/gate + norms → FP32; embed + lm_head → BF16; in-model vision/audio towers → BF16 passthrough.
• audio_tokenizer/ DROPPED; dflash/ drafter → BF16, separate (not counted in 170).
• Measured footprint (safetensors headers): experts MXFP4 + all-BF16 sensitive ≈ 164.7 GiB (≤170 GiB).

Files

• mimo_v2.py — PrismaQuant MiMoV2Profile (nested per-expert MoE, fused qkv, FP8-source, 3 dense MTP layers). Verified: detect_profile→MiMoV2Profile, 256 experts collapse per layer, dense-0 scored separately.
• mimo_v2.json — declarative model-structure spec.
• registry.patched.py — profile registration (before DeepseekV4 + DefaultProfile).
• calib/*.jsonl — domain calibration set (JSON/tool-call/tender-extraction), disjoint gate split.
• step1_keymatch.py — state-dict key-match diagnostic.

BLOCKER (documented for reproduction)

MiMo-V2.5-DFlash ships config-only (configuration_mimo_v2.py, no modeling_*.py; auto_map=AutoConfig only). transformers v5.0 built-in MiMoV2FlashForCausalLM (model_type mimo_v2_flash) is STRUCTURALLY INCOMPATIBLE: batched-3D experts (vs 72,192 separate per-expert tensors), split q/k/v (vs fused qkv_proj), no MTP. => A vendored modeling_mimo_v2.py (name-matching the checkpoint: nested 256-expert ModuleList w/ separate gate/up/down_proj; fused qkv_proj + attention_sink_bias on 39 SWA layers; model.mtp.layers.{0,1,2}; dense layer-0) is required for the transformers-based tooling path. MXFP4 being data-free also permits a direct tensor-level requant that bypasses model-loading. dflash/dflash.py may seed the MTP/attention shim.