Downloads · 30 days
120
10% of all-time downloads
Qwen3.8-27B-mlx-mxfp4-text-generation-only
randmaru/Qwen3.8-27B-mlx-mxfp4-text-generation-only
Qwen3.8-27B-mlx-mxfp4-text-generation-only is a text generation model from randmaru. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
This is an MXFP4 MLX quantization of Qwen/Qwen3.8-27B for Apple Silicon inference. [!Warning] Text-generation-only: all 333 vision weights in BF16 were removed.
All-time downloads
1.2K
Public
Parameters
26.9B
14.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.3 GB · 100%
Parameter types
How the weights are stored.
U3226.9B · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
randmaru/Qwen3.8-27B-mlx-mxfp4-text-generation-only
This is an MXFP4 MLX quantization of Qwen/Qwen3.8-27B for Apple Silicon inference.
[!Warning] Text-generation-only: all 333 vision weights in BF16 were removed.
MXFP4 vs 4Bit Quantization Comparison
| Parameter | MXFP4 | 4Bit |
|---|---|---|
| Quantization format | 4‑bit floating point with microscaling, group 32, shared exponent E8M0 | 4‑bit integer (INT4/NF4) |
| Tensor types | U8, U32, BF16 | BF16, U32 |
Parameter size (safetensors) | ~14.3 GB | ~15.13 GB |
| Total storage (all files) | ~14.32 GB | ~15.15 GB |
| Hardware support | Most efficient on GPUs with microscaling / FP8 tensor core support | Broad support, but often requires specialized INT4 kernels |
| Apple Silicon compatibility | Designed with hardware microscaling support in Apple Neural Engine / GPU | Works, but without specialized Neural Engine optimization |
| Inference speed | Higher on compatible hardware: FP path, lower dequantization overhead, higher throughput | Kernel‑dependent; usually lower or comparable at similar quality |
| Quality | Better preserves dynamic range, less degradation on outliers | Higher risk of accuracy loss on outliers at the same bitrate |
Key takeaways:
- Parameter size: The MXFP4 version has a smaller
safetensorsfootprint: ~14.3 GB vs ~15.13 GB for the 4-bit version. - Total storage: MXFP4 also occupies less total disk space: ~14.32 GB vs ~15.15 GB when summing all repository files.
- Vision weights removed: Both repositories are text-generation-only. All 333 vision weights in BF16 were removed to reduce size and focus inference on text generation.
- Performance: MXFP4 typically delivers higher inference throughput on hardware with microscaling/FP8 support, especially on Apple Silicon. The shared exponent per group of 32 elements reduces dequantization overhead and enables use of floating‑point tensor cores.
- Apple Silicon optimization: MXFP4 is designed with Apple Neural Engine / GPU microscaling support in mind, making it the recommended choice for MacBook.
- Quality: MXFP4 preserves dynamic range better, so generation quality can be higher at the same compression level.
[!Note] Actual speed depends on the backend, GPU, batch size, and quantization implementation.
- Task
- Text Generation
- Library
- mlx
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Aug 15, 2026
- Updated
- Aug 16, 2026