Downloads · 30 days
17
4% of all-time downloads
Qwen3.6-27B-MXFP4-MTP
Jim-and-HF/Qwen3.6-27B-MXFP4-MTP
Qwen3.6-27B-MXFP4-MTP is a image-text-to-text model from Jim-and-HF. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
This model extends mlx-community/Qwen3.6-27B-mxfp4 with native MTP (Multi-Token Prediction) heads stored in bfloat16 precision, enabling speculative decoding for ~1.5–2× faster generation.
All-time downloads
481
Public
Parameters
27.8B
16.1 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors16.1 GB · 100%
Parameter types
How the weights are stored.
U3226.9B · 97%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.6-27B-MXFP4-MTP
This model extends mlx-community/Qwen3.6-27B-mxfp4 with native MTP (Multi-Token Prediction) heads stored in bfloat16 precision, enabling speculative decoding for ~1.5–2× faster generation.
What's different from the base model
The base mlx-community mxfp4 conversion strips the MTP decoder layers. This checkpoint re-injects them:
| Component | Precision | Size |
|---|---|---|
| Main model (64 layers, vision encoder) | mxfp4 group_size=32 | ~15.0 GB |
| MTP decoder layer (×1) | bfloat16 (unquantized) | ~0.81 GB |
Total: ~15.8 GB.
MTP layer details
The MTP head consists of 1 decoder layer with self-attention, MLP, and projection:
| Tensor | Shape | Size |
|---|---|---|
mtp.layers.0.self_attn.q_proj | [12288, 5120] | 120 MB |
mtp.layers.0.mlp.gate_proj | [17408, 5120] | 170 MB |
mtp.layers.0.mlp.up_proj | [17408, 5120] | 170 MB |
mtp.layers.0.mlp.down_proj | [5120, 17408] | 170 MB |
mtp.fc (head projection) | [5120, 10240] | 100 MB |
| Other (norms, o/k/v proj) | — | ~80 MB |
| Total MTP | ~810 MB |
All MTP tensors are stored in bfloat16 — the overhead is small relative to the 15 GB main model, and keeping them unquantized preserves speculative decoding accuracy.
Model specs
- Architecture: Qwen3.6-27B (Qwen3.5 multimodal)
- Layers: 64 (16 full-attention, 48 linear-attention)
- Context: 262,144 tokens
- Vision: 27-layer ViT, image + video
- MTP: 1 decoder layer, shared embeddings
- Quantization: mxfp4 (OCP Microscaling FP4), group_size=32
- KV cache: turboquant 4-bit
Use with MLX
pip install -U mlx-vlm
python -m mlx_vlm.generate \
--model <hf-repo-id> \
--max-tokens 100 \
--temperature 0.0 \
--prompt "Describe this image." \
--image <path_to_image>
MTP speculative decoding
The MTP heads are already in the checkpoint — no separate drafter model or sidecar needed. Use with oMLX 0.39 (beta) which supports native Qwen3.6 MTP speculative decoding for ~1.5–2× faster generation.
Provenance
- Base weights:
mlx-community/Qwen3.6-27B-mxfp4 - MTP tensors: extracted from
Qwen/Qwen3.6-27Band merged in bf16
- Task
- Image-Text-to-Text
- Library
- mlx
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Jul 1, 2026
- Updated
- Jul 1, 2026