MiniMax-M2.5-REAP-139B-A10B-NVFP4

MiniMax-M2.5-REAP-139B-A10B — NVFP4 (W4A16)

NVFP4 weight-only (W4A16) quantization of Cerebras' MiniMax-M2.5-REAP-139B-A10B — a REAP (router-weighted expert-activation pruning) prune of MiniMax-M2 down to 139B total / 10B active (154 experts, pruned from 256; 62 layers). This is an agentic/reasoning MoE. The weights are quantized to NVFP4 (4-bit float E2M1, block-16 FP8-E4M3 micro-scale + FP32 per-tensor global scale, ≈4.5 bits/value) with BF16 activations, so it fits and serves on a single 128 GB DGX Spark / GB10. Unlike most REAP targets there is no BF16/FP16 source — the only upstream is block-FP8 (float8_e4m3fn + weight_scale_inv [128,128]); the shard-by-shard streaming quantizer dequantizes FP8→BF16 on the fly per shard (w_bf16 = w_fp8 × weight_scale_inv) before computing NVFP4, never materializing a giant intermediate (peak RAM ≈ one shard). Output is the standard compressed-tensors nvfp4-pack-quantized format — drop-in for vLLM, no custom modeling code (vLLM resolves MiniMaxM2ForCausalLM natively). On-disk size is 75 GB (down from 131 GB block-FP8); served runtime footprint ≈109 GB. Benchmarked on vLLM at prefill 128 / decode 120 tok/s (ShareGPT, concurrency 32, 64K context, 0 errors). The fused parallel projections (q/k/v→qkv, gate/up→gate_up, expert w1/w3) share one weight_global_scale per group — required for correct output under vLLM's fused MoE/attention layers.

Serving (vLLM)

vllm serve gauravmm/MiniMax-M2.5-REAP-139B-A10B-NVFP4 \
  --quantization compressed-tensors \
  --moe-backend marlin \
  --trust-remote-code \
  --max-model-len 65536 \
  --kv-cache-dtype fp8

--moe-backend marlin is mandatory on this NVFP4 W4A16 checkpoint — the flashinfer_cutlass MoE backend rejects W4A16 NVFP4 and crashes engine-core init.

Provenance

MiniMax-M2 (MiniMax AI, modified-MIT) → REAP prune (Cerebras) → NVFP4 W4A16 (this repo). License inherited from upstream (see LICENSE): MIT with a high-revenue attribution clause (display "MiniMax M2" if the deploying product exceeds the stated annual-recurring-revenue threshold).