Downloads · 30 days
50
16% of all-time downloads
MiniCPM5-2B-mlx-mxfp4
randmaru/MiniCPM5-2B-mlx-mxfp4
MiniCPM5-2B-mlx-mxfp4 is a text generation model from randmaru. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
This is an MXFP4 MLX quantization of openbmb/MiniCPM5-2B for Apple Silicon inference.
All-time downloads
322
Public
Parameters
2.5B
1.3 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors1.3 GB · 99%
Parameter types
How the weights are stored.
U322.5B · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
randmaru/MiniCPM5-2B-mlx-mxfp4
This is an MXFP4 MLX quantization of openbmb/MiniCPM5-2B for Apple Silicon inference.
MXFP4 vs 4Bit Quantization Comparison
| Parameter | MXFP4 | 4Bit |
|---|---|---|
| Quantization format | 4‑bit floating point with microscaling, group 32, shared exponent E8M0 | 4‑bit integer (INT4/NF4) |
| Tensor types | U8, U32, BF16 | BF16, U32 |
Parameter size (safetensors) | ~1.34 GB (1,337,358,111 bytes) | ~1.42 GB (1,416,035,480 bytes) |
| Total storage (all files) | ~1.35 GB (1,347,313,381 bytes) | ~1.43 GB (1,426,011,795 bytes) |
| Hardware support | Most efficient on GPUs with microscaling / FP8 tensor core support | Broad support, but often requires specialized INT4 kernels |
| Apple Silicon compatibility | Designed with hardware microscaling support in Apple Neural Engine / GPU | Works, but without specialized Neural Engine optimization |
| Inference speed | Higher on compatible hardware: FP path, lower dequantization overhead, higher throughput | Kernel‑dependent; usually lower or comparable at similar quality |
| Quality | Better preserves dynamic range, less degradation on outliers | Higher risk of accuracy loss on outliers at the same bitrate |
Key takeaways:
- Parameter size: The MXFP4 version has a slightly smaller
safetensorsfile (~1.34 GB vs ~1.42 GB). This reflects the actual file sizes from the respective repositories; the MXFP4 format with microscaling and floating‑point storage (U8/U32/BF16) results in a more compact representation in this case compared to the integer version (BF16/U32). - Total storage: MXFP4 also occupies less total disk space (~1.35 GB vs ~1.43 GB) when summing all files in the repository. This indicates more efficient packaging and/or fewer auxiliary files.
- Performance: MXFP4 typically delivers higher inference throughput (tokens/s) on hardware with microscaling/FP8 support, especially on Apple Silicon. The shared exponent per group of 32 elements reduces dequantization overhead and enables the use of floating‑point tensor cores, which can be faster than conventional INT4 kernels.
- Apple Silicon optimization: The MXFP4 format was developed with the hardware capabilities of Apple chips (M‑series) in mind, particularly the Neural Engine, which natively supports microscaling. Therefore, MXFP4 is the recommended choice for MacBook and offers the best balance of speed and quality on these devices.
- Quality: MXFP4 preserves the dynamic range of weights better, so generation quality can be higher at the same compression level.
Actual speed depends on the backend, GPU, batch size, and quantization implementation.
- Task
- Text Generation
- Library
- mlx
- Type
- llama
- License
- apache-2.0
- Created
- Sep 10, 2026
- Updated
- Sep 10, 2026