Qwen3.8-27B-MTP-MLX-bf16

Qwen3.8-27B-MTP-MLX-bf16

This is not a standalone model. It holds only the Multi-Token Prediction (MTP) drafter head, so it carries no token embeddings and no lm_head and it cannot generate text alone. Load it as the draft model beside a matching target, which supplies both.

Qwen ships this head inside Qwen/Qwen3.8-27B as 15 tensors under the mtp. prefix, but the MLX converter deletes them when it builds the main model, so the head is published separately and the runtime loads it as its own model with model_type: qwen3_5_mtp.

Use with mlx-vlm

mlx_vlm generate \
  --model vvsotnikov/Qwen3.8-27B-MLX-8bit \
  --draft-model vvsotnikov/Qwen3.8-27B-MTP-MLX-bf16 \
  --prompt "Write a quicksort in Python." \
  --max-tokens 256 --temperature 0.6 --enable-thinking

For local weights:

mlx_vlm generate \
  --model /path/to/target-model \
  --draft-model /path/to/Qwen3.8-27B-MTP-MLX-bf16 \
  --prompt "Write a quicksort in Python." \
  --max-tokens 256 --temperature 0.6 --enable-thinking

--draft-kind mtp is detected from model_type, so you do not need to pass it. The drafter proposes tokens each step while the target verifies them, and only accepted tokens reach the output, so quality follows the target rather than the drafter.

Model Details

• Model type: qwen3_5_mtp
• MTP block size: 3
• Target architecture: Qwen3.8-27B
• Precision: bfloat16, unquantized
• Runtime: MLX / mlx-vlm 0.6.8
• Format: Safetensors with MLX-compatible config and tokenizer files

How this was produced

python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split \
  --model Qwen/Qwen3.8-27B --output Qwen3.8-27B-MTP-MLX-bf16

The tool reads the 15 mtp. tensors out of the base checkpoint, strips that prefix, and adds 1.0 to every RMSNorm weight, because Qwen stores the norm scale minus one.

Verification

Pairing rules

Use a drafter and a target that come from ONE checkpoint, because the drafter binds to the target's embeddings at runtime, so a mismatched pair either fails a hidden-size check or drafts badly. This release publishes these drafters, so you can trade drafter size against acceptance:

• Qwen3.8-27B-MTP-MLX-bf16 <- this one
• Qwen3.8-27B-MTP-MLX-8bit
• Qwen3.8-27B-MTP-MLX-4bit

License and attribution

The weights derive from Qwen/Qwen3.8-27B under Apache 2.0, so the original license and its terms carry over. Read the license itself before you use this model.