Qwen3.8-27B-test-MTP-MLX-4bit

Qwen3.8-27B-test-MTP-MLX-4bit

Native MTP drafter extracted from the original BF16 checkpoint and quantized to 4-bit. It requires the matching main model and cannot generate independently. All 7 RMSNorm offsets were verified against the source.

Source: vvsotnikov/Qwen3.8-27B-test, revision 13171b198fbeb3a2ecc5aaabe0d3ffd2fc8a39fa. The source is a 50/50 linear merge of Qwen3.6-27B and Qwen3.8-27B. See merge-manifest.json for the original merge provenance and conversion-manifest.json for conversion settings and runtime versions.

Matching pair:

• Main model
• MTP drafter

Usage

python -m mlx_vlm generate \
  --model vvsotnikov/Qwen3.8-27B-test-MLX-4bit \
  --draft-model vvsotnikov/Qwen3.8-27B-test-MTP-MLX-4bit \
  --draft-kind mtp \
  --prompt "Write a Python function that merges two sorted lists." \
  --max-tokens 2048 --enable-thinking --temperature 1.0 \
  --gen-kwargs '{"top_p":0.95,"top_k":20}'

Validated with MLX-VLM 0.6.15 (revision 20eec6cb5564c6a196b046d869d2081c29e3ff92), MLX 0.32.0, and Transformers 5.14.0 on an Apple M5 Max.

Validation

Structural checks passed for quantization, tensor inventory, main/drafter compatibility, and MTP norm offsets. A text smoke test generated a complete Python answer (1,863 tokens, 82.1% draft acceptance). An image smoke test correctly identified a red square and blue circle (234 tokens, 83.9% draft acceptance). Both used thinking enabled, temperature 1.0, top-p 0.95, top-k 20, and seed 42. These are smoke checks, not a quality benchmark or a comparison against BF16.