MiniMAX-M2.7-MXFP4-CANN

MiniMAX-M2.7-MXFP4-CANN

MXFP4 (W4A4) quantized MiniMax-M2.7 for Ascend CANN / vLLM-Ascend.

MiniMax-M2.7 的 MXFP4 量化版本，面向昇腾 NPU（CANN）推理部署。对 Attention QKV 与 MoE Expert 权重进行 MXFP4 量化，采用 PTQ（SmoothQuant OSPlus）+ QAT 两阶段方案，量化全程在 昇腾 910C 上完成。

在推荐推理配置下（VLLM_MXFP4_ACT_QDQ_SCALE_FACTOR=5.8），7 项评测任务相对 BF16 基线的平均性能保有率为 99.3%。

• Base model: MiniMaxAI/MiniMax-M2.7
• Quantization: MXFP4 W4A4（Attention QKV + MoE Experts）
• Algorithm: PTQ (SmoothQuant OSPlus) → QAT
• Quantization hardware: Ascend 910C
• Inference: cann-recipes-infer / vLLM MiniMax MXFP4

Model Details

This checkpoint is intended for Ascend NPU serving via the official CANN vLLM recipe. It is not a drop-in replacement for generic CUDA vLLM MXFP4 loaders.

Quantization

Quantization is a two-stage pipeline, executed end-to-end on Ascend 910C:

1. PTQ — SmoothQuant OSPlus
   Search per-layer smoothing scales on calibration data, fuse the scales into BF16 weights, then convert fused weights to MXFP4 (RTN, block size 32). This reduces activation outliers before 4-bit quantization.

2. QAT
   Quantization-aware training on the PTQ checkpoint to recover accuracy lost in the 4-bit conversion, especially on knowledge-intensive and long-context tasks.

What is quantized

• Attention: q_proj / k_proj / v_proj
• MoE: expert FFN weights (W4A4 MXFP4)

Typical high-precision leftovers (kept closer to original precision): embeddings, RMSNorm, router / gate, and lm_head.

At serve time, weights are loaded from the MXFP4 (Quark-style) checkpoint; activations are quantized online with MXFP4 QDQ. The vLLM-Ascend recipe can additionally apply KV-cache MXFP4 QDQ (token-wise, group_size=32, with the first VLLM_KV_MXFP4_ANCHOR tokens kept in BF16) plus block-diagonal Hadamard rotation on Q/K.

Evaluation

Evaluated with the cann-recipes-infer MiniMax MXFP4 vLLM recipe and:

VLLM_MXFP4_ACT_QDQ_SCALE_FACTOR=5.8

VLLM_MXFP4_ACT_QDQ_SCALE_FACTOR is the block-scale divisor used in activation MXFP4 QDQ:

scale = 2^round(log2(max_abs / scale_factor))

5.8 is the recommended setting for this checkpoint (MXFP4 E2M1 codebook max is 6.0; 5.8 is a slight tightening that we found preserves accuracy better than the default 6.0 on this model).

Average retention across the seven tasks is 99.3% of the BF16 baseline. LiveCodeBench is slightly above the baseline (sampling variance is possible on generation benchmarks).

How to Use

Follow the official CANN recipe (MiniMax-M2.5 and MiniMax-M2.7 share the same architecture; point MODEL_PATH at this repo):

Recipe: https://gitcode.com/cann/cann-recipes-infer/tree/master/integration/vllm/minimax_m2.5_mxfp4

Hardware

Recommended image

docker pull quay.io/ascend/vllm-ascend:v0.18.0rc1-a3

Serve

cd cann-recipes-infer/integration/vllm/minimax_m2.5_mxfp4
source set_env.sh
bash patch_vllm/apply.sh

MODEL_PATH=/path/to/guanwenyu1995/MiniMAX-M2.7-MXFP4-CANN \
SERVED_MODEL_NAME=MiniMAX-M2.7-MXFP4-CANN \
TP_SIZE=16 \
VLLM_MXFP4_ACT_QDQ_SCALE_FACTOR=5.8 \
VLLM_KV_MXFP4_ANCHOR=32 \
bash run_vllm_w4a4.sh

VLLM_MXFP4_ACT_QDQ_SCALE_FACTOR=5.8 is required to reproduce the accuracy numbers above. Do not omit it if you care about the reported retention.

Sanity check

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMAX-M2.7-MXFP4-CANN",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 256
  }'

Decoding parameters

Inherited from MiniMax-M2.7:

• temperature=1.0
• top_p=0.95
• top_k=40

Default system prompt:

You are a helpful assistant. Your name is MiniMax-M2.7 and is built by MiniMax.

More environment variables (TP_SIZE, KV-cache MXFP4 switches, FlashComm, tool/reasoning parsers, etc.) are documented in the recipe README.

Intended Use & Limitations

• Intended use: offline / online inference of MiniMax-M2.7 on Ascend NPUs with vLLM-Ascend, including coding, math, QA, and long-context workloads covered by the table above.
• Not intended: training from this MXFP4 checkpoint; serving on non-Ascend backends without the CANN vLLM patches.
• Limitations: 4-bit W4A4 still shows a larger gap on GPQA Diamond (97.1% retention) and LongBench v2 (97.3% retention) than on GSM8K / MATH500. Generation benchmarks can fluctuate with sampling.

License

This repository is a quantized derivative of MiniMaxAI/MiniMax-M2.7. Use of the weights is subject to the original MiniMax license:

https://github.com/MiniMax-AI/MiniMax-M2.7/blob/main/LICENSE

Acknowledgements

• Base model: MiniMaxAI/MiniMax-M2.7
• Inference recipe: cann-recipes-infer (vLLM / vLLM-Ascend MXFP4 patches)