MiniMax-M2.7-MXFP4

MiniMax-M2.7-MXFP4

MXFP4 quantization of MiniMax-M2.7 (228B params, 62 layers, 256 experts/layer, top-8 sigmoid routing).

All 15,872 MoE expert weights quantized to MXFP4. Attention, layer norms, embeddings, and router weights kept at original precision.

Quality Analysis

KLD is heavily right-skewed: median KLD is 0.031 nats/tok (5.6x lower than the mean). 96.6% of tokens have KLD < 1 nat. Only 69 out of 2048 eval tokens show significant divergence -- these are low-confidence positions where the model is already distributing probability across many candidates.

Error is diffuse across experts: per-expert quantization error analysis of all 15,872 experts shows extremely uniform error (std=0.000271, range 0.110--0.116). The 256-expert top-8 architecture is inherently quantization-tolerant -- each expert contributes ~1/8th of the output, so MXFP4 errors average out across the mixture.

Format

MXFP4 block-32 quantization in compressed-tensors format:

• weight_packed: uint8 [out, in//2] -- two 4-bit values packed per byte (even=low nibble, odd=high nibble)
• weight_scale: uint8 e8m0 [out, in//32] -- one shared exponent per block of 32 elements

Quantization is calibration-free (no calibration data needed). MXFP4 block-32 scaling is deterministic -- the shared exponent is derived directly from the max magnitude in each block.

Quantized with quant4.

Serving

vLLM

Requires vLLM with MXFP4 compressed-tensors support and the CUTLASS FP4xFP8 kernel for Blackwell GPUs:

vllm serve /path/to/MiniMax-M2.7-MXFP4 \
    --tensor-parallel-size 2 \
    --trust-remote-code \
    --max-num-seqs 512 \
    --enable-chunked-prefill \
    --max-num-batched-tokens 16384 \
    --kv-cache-dtype fp8

Memory Budget

At 119 GB, this fits on 2x DGX Spark (2x 120 GB = 240 GB total) with ~100 GB remaining for KV cache, enabling long-context or multi-session serving that would be impossible with the 215 GB FP8 original.

Evaluation Details

Evaluated on WikiText-2 test set (2048 tokens) using layer-by-layer streaming inference with MiniMaxLayerRunner. Both models run identical forward passes; logits compared token-by-token.

Acknowledgments

Based on MiniMax-M2.7 by MiniMax. Original model license applies.