Qwen3.8-27B-AEON-Ultimate-MLX-MTP-Drafter

Qwen3.8-27B-AEON MTP Drafter (MLX)

The native multi-token-prediction head of AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 (revision 8f76e82), split into the standalone MLX drafter format that mlx-vlm expects for mtp speculative decoding. 15 tensors, BF16, 810 MB, block_size: 3.

AEON's BF16 grafts the MTP head back from stock Qwen3.8 (15 tensors, hash-matched), so this drafter is the original Qwen-trained head — not retrained.

Companion to VisualInference/Qwen3.8-27B-AEON-Ultimate-Multimodal-MLX-6bit; works with any MLX quantization of the same base.

Status (2026-09-10)

This drafter still works as documented, and --draft-block-size 3 is still the right setting for it. Two measured facts since publication, for anyone choosing a speculative-decoding setup on a Mac:

• On a 6-bit Qwen3.8-27B body at 24-27k tokens of agent context, mlx-vlm with a native MTP drafter measured 6-7 tok/s end to end, while mlx-dspark with incoai/Qwen3.8-27B-DFlash2 measured 15-19 tok/s on the same body and prompts. Both are lossless. At short context the gap is smaller (this card's numbers were measured at up to 13k).
• DFlash2 is trained on stock Qwen3.8 and works unchanged with any abliterated Qwen3.8-27B body, including AEON's; it changes speed, never output.

The lane recipe, launcher settings and patches: qwen38-27b-mlx-agent-lane.

How it was made

python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split \
  --model AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 \
  --output Qwen3.8-27B-AEON-Ultimate-MLX-MTP-Drafter

(Use the official split tool — hand-extracting mtp.* tensors produces a broken drafter with 0% acceptance: the tool applies the RMSNorm weight-convention shift and stamps format: mlx metadata.)

Usage

python -m mlx_vlm generate \
  --model VisualInference/Qwen3.8-27B-AEON-Ultimate-Multimodal-MLX-6bit \
  --draft-model VisualInference/Qwen3.8-27B-AEON-Ultimate-MLX-MTP-Drafter \
  --draft-kind mtp --draft-block-size 3 \
  --prompt "..."

Speculative decoding is lossless — rejected drafts fall back to the target model's own tokens.

Measured effect

Test machine: Mac mini M4 Pro, 48 GB, with the companion 6-bit target model. Speedups carry across chips since the drafter's relative overhead is small.

Block size 3 is the all-round sweet spot; block ≥5 regresses (draft accuracy decays past 2–3 tokens). Acceptance ~46% on open prose, higher on structured output.