MiMo-V2.5-Text-Only-oQ4-MLX

MiMo-V2.5-Text-Only-oQ4-MLX

A text-only, mixed-precision MLX quantization of XiaomiMiMo/MiMo-V2.5, produced with oMLX's oQ4 quantizer.

MiMo-V2.5 is a sparse Mixture-of-Experts model with 310B total and 15B activated parameters, a hybrid sliding-window and global attention stack, and a 1M token context window. This artifact contains the language backbone only.

Text-only

The vendored mlx-lm MiMo-V2 implementation exposes the language backbone only, so the vision tower, audio encoder, speech embeddings and MTP heads are dropped during conversion. Verified against model.safetensors.index.json:

This model cannot process images, video or audio, and it carries no speculative-decoding heads. If you need the omnimodal capabilities, use the upstream checkpoint with a runtime that supports them.

config.json still carries some multimodal metadata (vision_model_type, processor_config) and a preprocessor_config.json is present, both inherited from the base checkpoint. No weights back them.

Quantization profile

oQ is sensitivity-driven mixed precision rather than a uniform cast. The base grid is 4-bit, group_size 64, affine, with 197 modules promoted to 8-bit.

Keeping every attention projection and the embedding and output layers at 8-bit is where most of the extra budget goes. The MoE expert weights carry the bulk of the parameters and are the part that is quantized to 4 bits.

The sensitivity map used for this conversion was measured, not inferred from layer position. The source is a native FP8 checkpoint whose bf16 calibration footprint (about 587 GB) exceeds what fits alongside the quantizer, so oMLX auto-built a uniform 4-bit proxy on disk and measured sensitivity against that:

oQ4: calibration footprint (586.8 GB) exceeds 75% of calibration capacity
(488.0 GB; limit=366.0 GB, system available=489.9 GB, Metal available=488.0 GB).
Full-model calibration will use a proxy.

Attention layout

MiMo-V2.5 stores attention as a single fused qkv_proj that is already sharded for tensor parallelism, with a block-128 weight_scale_inv whose padding is interleaved per shard rather than trailing. Dequantizing that as one block grid crosses shard boundaries and silently produces wrong weights on the 9 global attention layers, which are the only ones that carry padding.

This artifact was produced with a shard-aware dequant, and the fused tensors are fully resolved into split projections: 0 remaining qkv_proj tensors, and 48 each of q_proj, k_proj, v_proj and o_proj.

Requirements

mlx-lm does not yet ship a MiMo-V2 implementation upstream (ml-explore/mlx-lm#1219 is open), so this checkpoint needs a runtime that provides it. oMLX vendors that patch and is the runtime this artifact was built and tested with.

Apple silicon with enough unified memory to hold 164.59 GiB of weights plus KV cache. This was built and tested on a 512 GB M3 Ultra.

config.json declares an auto_map, so loading through Transformers would require trust_remote_code=True. The MLX path resolves the model class from the runtime instead.

Reproducing

Built with oMLX v0.5.4.dev1 plus two fixes that this conversion depends on:

• jundot/omlx#2417 budgets FP8 calibration on the bf16 footprint, and dequantizes the fused QKV per tensor parallel shard. Without the budget fix the proxy is never built and the process is killed partway through calibration. Without the dequant fix conversion aborts on the 9 global attention layers.
• jundot/omlx#2416 routes text-only multimodal bases through mlx-lm. MiMo-V2.5 ships a vision_config, so without this the calibration load is handed to mlx-vlm and fails.

Verification

• Converts, loads and generates coherent output including a reasoning trace.
• No vision, audio, speech or MTP tensors remain, and no fused qkv_proj tensors remain (both checked against the tensor index).
• Per-module precision assignments in config.json reconcile exactly with the scale and bias tensor counts in the index: 338 quantized modules, 197 at 8-bit and 141 at 4-bit.

No benchmark scores are claimed here. Quantization changes model behaviour, and the upstream evaluation numbers do not transfer to a 4-bit derivative.

License and attribution

The base model is MIT licensed and this derivative follows the same license. Review the upstream model card for intended use and limitations.

This is an unofficial community quantization. It is not affiliated with, sponsored by, or endorsed by Xiaomi.