Downloads · 30 days
12.2K
19% of all-time downloads
Qwen3.8-27B-MTP-4bit
mlx-community/Qwen3.8-27B-MTP-4bit
Qwen3.8-27B-MTP-4bit is a text generation model from mlx-community. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
This repository contains the Multi-Token Prediction (MTP) drafter weights split from Qwen/Qwen3.8-27B and quantized with mlxvlm.convert for mlx-vlm speculative decoding.
All-time downloads
62.7K
Public
Parameters
425M
259 MB on disk
Likes
41
Trending 5
Repo makeup
Click a slice to open those files.
.safetensors239 MB · 90%
Parameter types
How the weights are stored.
U32425M · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.8-27B-MTP-4bit
This repository contains the Multi-Token Prediction (MTP) drafter weights split from Qwen/Qwen3.8-27B and quantized with mlx_vlm.convert for mlx-vlm speculative decoding.
This is not a standalone model. Use it as the draft model alongside a compatible Qwen3.8 27B target checkpoint.
Use with mlx-vlm
mlx_vlm generate \
--model mlx-community/Qwen3.8-27B-4bit \
--draft-model mlx-community/Qwen3.8-27B-MTP-4bit \
--prompt "Write a quicksort in Python." \
--max-tokens 256 \
--enable-thinking
--draft-kind mtp is detected automatically from model_type.
Model details
- Model type:
qwen3_5_mtp - MTP block size: 3
- Target architecture: Qwen3.8 27B
- Precision: MLX affine 4-bit, group size 64
- Runtime: MLX /
mlx-vlm - Source revision:
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
The adapter holds only the MTP drafter weights; the target supplies the token embeddings and language-model head at runtime. Use an adapter and target derived from the same Qwen3.8 27B checkpoint. The upstream Apache 2.0 license and model limitations continue to apply.
- Task
- Text Generation
- Library
- mlx
- Type
- qwen3_5_mtp
- License
- apache-2.0
- Languages
- en
- Created
- Aug 14, 2026
- Updated
- Aug 14, 2026