Downloads · 30 days
38
6% of all-time downloads
qwen3.8-27b-it-oQ4e-mtp
dynamicagency/qwen3.8-27b-it-oQ4e-mtp
qwen3.8-27b-it-oQ4e-mtp is a machine learning model from dynamicagency. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlx.
oQ 4-bit mixed-precision quantization of Qwen/Qwen3.8-27B (oMLX v0.5.7, group size 64), with the checkpoint's native MTP head preserved for oMLX's Lightning MTP.
All-time downloads
621
Public
Parameters
27.8B
17 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors17 GB · 100%
Parameter types
How the weights are stored.
U3227.3B · 98%
Model card
From the Hugging Face model README
Qwen3.8-27B-oQ4e-mtp
oQ 4-bit mixed-precision quantization of Qwen/Qwen3.8-27B (oMLX v0.5.7, group size 64), with the checkpoint's native MTP head preserved for oMLX's Lightning MTP.
17 GB, MLX safetensors. Tested on an Apple M3 Ultra (256 GB).
Speed
Lightning MTP only, vs. also enabling SpecPrefill and TurboQuant KV:
| Context | PP tok/s (before → after) | TG tok/s (before → after) | Peak mem (before → after) |
|---|---|---|---|
| 1k | 320.0 → 318.7 | 49.3 → 46.6 | 18.4 → 18.7 GB |
| 4k | 333.8 → 332.4 | 47.9 → 49.1 | 19.6 → 20.0 GB |
| 8k | 332.1 → 331.2 | 52.5 → 49.4 | 19.8 → 20.2 GB |
| 16k | 320.9 → 1,368 | 52.7 → 64.5 | 20.5 → 19.8 GB |
| 32k | 297.6 → 1,332 | 48.9 → 62.9 | 21.7 → 20.1 GB |
| 64k | 258.6 → 1,204 | 34.7 → 51.2 | 24.3 → 20.6 GB |
| 128k | 204.7 → 1,006 | 26.8 → 50.9 | 29.5 → 21.9 GB |
| 195k | 167.8 → 857.2 | 20.0 → 40.8 | 34.8 → 24.3 GB |
Recommended: Lightning MTP + SpecPrefill (small same-tokenizer-family draft model, e.g. Qwen3.5-0.8B) + TurboQuant KV (4-bit).
Intelligence
Same two configs, oMLX's built-in accuracy suite:
| Benchmark | MTP only | + SpecPrefill + TurboQuant |
|---|---|---|
| MMLU (1000 q) | 82.5% | 82.9% |
| GSM8K (100 q) | 90.0% | 88.0% |
| HumanEval (164 q) | 94.5% | 93.9% |
| MBPP (200 q) | 81.5% | 84.5% |
| Aggregate | 84.2% | 84.7% |
Quantization details
- Model type: qwen3_5
- Bits: 4 (oQ4e)
- Group size: 64
- Format: MLX safetensors
- MTP head: preserved, usable via oMLX's
mtp_enabled
Usage
pip install huggingface_hub[hf_xet]
huggingface-cli download --local-dir qwen3.8-27b-it-oQ4e-mtp dynamicagency/qwen3.8-27b-it-oQ4e-mtp
- Library
- mlx
- Type
- qwen3_5
- Created
- Aug 14, 2026
- Updated
- Aug 17, 2026