Qwen3.8-27B-test-MLX-4bit

Qwen3.8-27B-test-MLX-4bit

4-bit MLX conversion of the main model. The language model uses affine RTN quantization with group size 64. The vision tower remains dense BF16. The native MTP head is provided separately.

Source: vvsotnikov/Qwen3.8-27B-test, revision 13171b198fbeb3a2ecc5aaabe0d3ffd2fc8a39fa. The source is a 50/50 linear merge of Qwen3.6-27B and Qwen3.8-27B. See merge-manifest.json for the original merge provenance and conversion-manifest.json for conversion settings and runtime versions.

Matching pair:

• Main model
• MTP drafter

Usage

python -m mlx_vlm generate \
  --model vvsotnikov/Qwen3.8-27B-test-MLX-4bit \
  --draft-model vvsotnikov/Qwen3.8-27B-test-MTP-MLX-4bit \
  --draft-kind mtp \
  --prompt "Write a Python function that merges two sorted lists." \
  --max-tokens 2048 --enable-thinking --temperature 1.0 \
  --gen-kwargs '{"top_p":0.95,"top_k":20}'

Validated with MLX-VLM 0.6.15 (revision 20eec6cb5564c6a196b046d869d2081c29e3ff92), MLX 0.32.0, and Transformers 5.14.0 on an Apple M5 Max.

Validation

Structural checks passed for quantization, tensor inventory, main/drafter compatibility, and MTP norm offsets. A text smoke test generated a complete Python answer (1,863 tokens, 82.1% draft acceptance). An image smoke test correctly identified a red square and blue circle (234 tokens, 83.9% draft acceptance). Both used thinking enabled, temperature 1.0, top-p 0.95, top-k 20, and seed 42. These are smoke checks, not a quality benchmark or a comparison against BF16.