MiMo-V2.5-MLX

MiMo-V2.5 — text-only, packaged for MLX

This is a convenience repackaging of XiaomiMiMo/MiMo-V2.5 for inference on Apple Silicon via MLX.

Every individual weight value is bit-identical to the upstream release. This repository repacks the 32 source safetensors shards into a single file, stacks per-expert MoE weights along a new leading axis (required by MLX MoE loaders), and excludes multimodal weights (vision, audio, MTP heads) since this targets the text-only inference path.

Validated on M3 Ultra 512 GB: 30.0 tok/s decode (warm), ~535 tok/s prefill at L=2048 — running on stock upstream MLX (no fork) plus the mlx-block-fp8 extension.

Setup

MiMo-V2.5 uses the block_fp8 quantization scheme (DeepSeek-V3-style: unpacked E4M3 codes + 2D FP32 scales), which is not built into MLX. The kernels for it ship as a standalone MLX extension that builds against stock upstream MLX — no MLX fork required.

1. MLX (built shared)

Build MLX from upstream with BUILD_SHARED_LIBS=ON (required — a static build causes the extension to get a duplicate runtime and silently produce zeros):

git clone https://github.com/ml-explore/mlx.git
cd mlx
CMAKE_ARGS="-DBUILD_SHARED_LIBS=ON" \
  python3 -m pip install -e . --no-build-isolation

2. The mlx-block-fp8 extension

The block_fp8 matmul + MoE kernels:

git clone https://github.com/yohann-bearzi/mlx-block-fp8.git
cd mlx-block-fp8
python3 -m pip install -e . --no-build-isolation

3. mlx_lm with mimo_v2 base support

MiMo-V2 base model support (until merged upstream): ml-explore/mlx-lm#1219. Install mlx_lm from that branch, or copy mimo_v2.py + switch_layers.py from it into your mlx_lm/models/ directory.

4. The block_fp8 model class (shipped in this repo)

Copy mimo_v2_block_fp8.py into your mlx_lm models directory:

cp mimo_v2_block_fp8.py $(python3 -c 'import mlx_lm, os; print(os.path.dirname(mlx_lm.__file__))')/models/

5. (Optional, +~3%) the SDPA fix

For full decode throughput, apply ml-explore/mlx#3637, which enables MLX’s fused attention kernel for MiMo’s asymmetric Q/V head dims (192/128). Without it, attention uses a slower compiled-graph fallback.

Usage

import json, mlx.core as mx
import mlx_lm.utils as U

mx.set_wired_limit(300 * 1024**3)  # M3 Ultra 512GB

cfg = json.load(open("config.json"))
cfg["model_type"] = "mimo_v2_block_fp8"
mc, ac = U._get_classes(cfg)
m = mc(ac.from_dict(cfg))

w = mx.load("mimo_v2.5_block_fp8.safetensors")
w = m.sanitize_block_fp8(w)
m.apply_block_fp8(w)
del w
import gc; gc.collect()
mx.eval(m.parameters())

Reproducing from source

If you’d rather convert from XiaomiMiMo’s release yourself, this repo ships the converter script. Download Xiaomi’s shards, then:

python3 convert_mimo.py --src /path/to/XiaomiMiMo/MiMo-V2.5 \
                        --out /path/to/output/mimo_v2.5_block_fp8.safetensors

The converter does two things only:

1. Concatenates the 32 source safetensors shards into a single file.
2. Stacks the 256 per-expert MoE weights per layer into one tensor with shape [256, …] per projection (required by MLX MoE loaders).

No quantization, permutation, scale manipulation, or padding is applied. Conversion takes ~10 minutes on a fast SSD.

What’s in this repo