Qwen3.8-27B-MLX-bf16-mtp

Qwen3.8-27B-MLX-bf16-mtp

Full-precision bf16 MLX conversion of Qwen/Qwen3.8-27B with the native MTP (multi-token prediction) head preserved, built for the oMLX inference engine on Apple Silicon.

• Complete VLM: vision tower included (image + video understanding).
• MTP-preserved: all 15 mtp.* tensors kept (mtp_num_hidden_layers: 1), enabling Lightning-MTP speculative decoding in oMLX (mtp_enabled: true).
• bf16 everywhere (~52 GB weights): no quantization; the Gated DeltaNet recurrent state self-promotes to fp32 internally at runtime, so bf16 storage is lossless in practice.
• Norm convention: weights are in MLX's shifted (+1) RMSNorm convention, including all MTP-head norms — safe for oMLX's per-key convention detection (no double shift).

Conversion details

• Converter: mlx_vlm.convert (mlx-vlm 0.6.3, mlx 0.32.0) run with oMLX 0.5.7's mlx_vlm_mtp patches applied (apply_mlx_vlm_mtp_runtime_patch() + apply_mlx_vlm_mtp_patch() before load), which is what preserves the MTP head — stock mlx_vlm.convert strips mtp.*.
• dtype="bfloat16", quantize=False.
• Source revision: 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0.

Measured on an M5 Max (128 GB), oMLX 0.5.7

31K-token prompt prefills in ~40 s (GDN chunked Metal kernel path); needle retrieval and image grounding verified.

Usage with oMLX

Place under ~/.omlx/models/<org>/Qwen3.8-27B-MLX-bf16-mtp (or download via the oMLX admin dashboard), then enable MTP for the model:

"Qwen3.8-27B-MLX-bf16-mtp": { "mtp_enabled": true, "max_context_window": 262144 }

oMLX routes it to the VLM engine automatically and selects the Lightning MTP speculative backend. Sampling defaults from generation_config.json: temperature 1.0, top_p 0.95, top_k 20.

If you want a standalone drafter for oMLX's external vlm_mtp_enabled path instead, you can extract one from this checkpoint's source with python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split (emits model_type: qwen3_5_mtp, block_size: 3, ~830 MB). The in-checkpoint native path above is faster and is the recommended default.

The model also works with stock mlx-vlm ≥ 0.6.3 as a plain bf16 VLM (the mtp.* tensors are ignored by loaders that don't implement MTP).