Qwen3.8-27B-8bit-MTP-MLX

Qwen3.8-27B 8-bit + MTP head (MLX)

mlx-community/Qwen3.8-27B-8bit plus the model's native multi-token-prediction head, so MLX runtimes with native MTP speculative decoding (oMLX mtp_enabled, mlx-lm PR 990 lineage) decode ~2.5–3× faster. Weights are otherwise byte-identical to the mlx-community conversion.

Full A/B and method: MTP.md · code: bluehawana/Qwen3.827B-SGLang-mpbm5max — Track D.

What was added

model-mtp.safetensors — the 15 language_model.mtp.* tensors (quantized 8-bit/gs64 like the body), taken from EigenLabs/Qwen3.8-27B-MTP-bf16 (bit-identical to Qwen/Qwen3.8-27B's own head; norms already in MLX +1 convention), registered in model.safetensors.index.json; text_config.mtp_num_hidden_layers = 1. The mlx-community conversions declare the head in config.json but strip its weights, which silently disables MTP.

Use with oMLX

1. Download into your oMLX model dir (or HF cache).
2. Admin UI → model settings → Lightning MTP on (API: PUT /admin/api/models/<id>/settings {"mtp_enabled": true}), load. The log shows Speculative backend selected and per-request MTP[...] accept=… lines.
3. Optional for Qwen3.8: set chat_template_kwargs.reasoning_effort to medium/low — the template defaults to xhigh.

Draft acceptance on coding/agent traffic: 75–99 %, ~3 tokens per verify pass (depth 3). MTP is single-stream; concurrent requests fall back to plain batched decode.

Size: ~29.3 GB. Credits: Qwen team (model, Apache-2.0), mlx-community (quantization), EigenLabs / Layr-Labs MTP challenge (head export), oMLX (runtime).