MiniMax-M2.5-REAP-139B-A10B-MLX-3.7bit

MiniMax-M2.5-REAP-139B-A10B-MLX-3.7bit

🚧 Experimental — Quality Degraded. This is an experimental quantization with known quality issues. The aggressive 3-bit compression causes significant performance loss in reasoning tasks and emoji/special character corruption. Released for research purposes only — not recommended for production use.

Mixed-precision MLX quantization of MiniMax-M2.5-REAP-139B-A10B — Cerebras' REAP-pruned MiniMax-M2.5 with 154 experts (pruned from 256), optimized for Apple Silicon local inference.

• 3.676 BPW | 60 GB | 139B total / 10B active per token

⚠️ Known Limitations

This quantization is released as an experimental / research artifact. The aggressive 3-bit compression of the expert FFN layers (93.9% of total parameters) results in noticeable quality degradation in complex reasoning scenarios. Users should be aware of the following:

Observed Quality Issues

Based on a standardized 10-question benchmark covering debugging, algorithms, security, concurrency, system design, and logic reasoning:

Benchmark Score

The quality issues are concentrated in high-difficulty questions requiring deep understanding and rigorous reasoning, suggesting that MiniMax-M2.5's expert FFN layers are particularly sensitive to sub-4-bit quantization. The model remains capable for straightforward tasks (algorithms, code transformation, basic Q&A scored full marks).

Research Value

Despite the quality limitations, this quantization may have some research value:

1. Feasibility demonstration: Proves that 139B MoE models can physically fit on 96GB Apple Silicon devices
2. Sensitivity analysis: Reveals that MiniMax' architecture is more quantization-sensitive than Qwen3.5's, particularly in expert FFN layers
3. Architecture comparison: While Qwen3.5-122B at 3.7bit retains 95/100 reasoning capability, MiniMax-139B at the same bit-rate drops to 71/100 — suggesting that quantization resilience varies significantly across architectures
4. Baseline for future work: Serves as a reference point for evaluating improved quantization techniques (GPTQ, AWQ, or calibration-based methods) on this architecture

📊 Benchmark (M2 Max 96GB)

While the reasoning quality is significantly impacted, the inference speed remains impressive for a 139B class model on local hardware:

Tested on M2 Max (38c GPU) with 96GB Unified Memory.

🚀 Hardware Requirements

The original FP8 model weighs 131GB, and BF16 would be an impractical 260GB. Through mixed-precision quantization, we've compressed it to 60GB:

• 96GB Unified Memory (Minimum): Fits the full model with moderate headroom for KV cache, enabling 139B-parameter local inference on a single Mac Studio or MacBook Pro.
• 128GB+ Unified Memory (Recommended): Delivers comfortable headroom for the KV cache, supporting longer context windows.

Quantization

4-tier mixed precision by functional sensitivity:

Optimization Philosophy

Three targeted upgrades over baseline 3-bit quantization, costing only ~1.4GB total:

1. embed/lm_head → 8-bit (+0.3GB): Every token passes through these layers. Quantization noise here corrupts all downstream computation (embed) or directly distorts output probabilities (lm_head).
2. All attention → 8-bit (+0.6GB): Attention is only 2% of parameters. Going from 3→8bit costs a trivial 0.6GB but provides near-lossless attention quality, effectively preventing repetition and degradation.
3. Entry/exit layer experts → 4-bit (+0.5GB): Layer 0's quantization errors propagate through all 61 subsequent layers. Layer 61's output feeds directly to lm_head with no opportunity for error correction.

Model Architecture

Usage

from mlx_lm import load, generate

model, tokenizer = load("MoringLabs/MiniMax-M2.5-REAP-139B-A10B-MLX-3.7bit")

response = generate(model, tokenizer, prompt="Hello!", max_tokens=200)
print(response)

Original Model

This quantization is based on cerebras/MiniMax-M2.5-REAP-139B-A10B, which was created by applying REAP (Router-weighted Expert Activation Pruning) to MiniMaxAI/MiniMax-M2.5 with a 40% pruning rate, reducing experts from 256 to 154 per layer while maintaining near-lossless performance.

License

Modified MIT (inherited from base model)