Qwen3.6-27B-MLX-3bit

Qwen3.6-27B-MLX-3bit

Summary

MLX 3bit (affine, 3-bit, group size 64) quantization of Qwen/Qwen3.6-27B at upstream revision 6a9e13bd6fc8f0983b9b99948120bc37f49c13e9, produced directly from the BF16 safetensors with pipelines.mlx_direct_quantize (majek repo). ~12.0 GiB on disk.

Runtime caveat: this is a weights + config pack published ahead of runtime availability; the qwen3_5 vision-language architecture is now supported upstream (mlx-lm PR #1345, merged 2026-06-04; mlx-vlm support available). No local inference smoke was performed on these artifacts, so the quality of this variant is not verified.

Why this variant

Affine 3-bit, group size 64 — small footprint with elevated quality risk (unverified; see the runtime caveat).

Reproduce

python -m pipelines.mlx_direct_quantize --model qwen3.6-27b --base-dir /tmp/mlx-direct-release/qwen3.6-27b/base --out-dir /tmp/mlx-direct-release/qwen3.6-27b/3bit --bits 3 --mode affine --group-size 64

Vision tower

The model.visual.* tower (333 tensors) is passed through unquantized in BF16 — only the text tower (model.language_model.*, lm_head.*) is quantized.

Family

All MLX variants of this model ship together:

• majentik/Qwen3.6-27B-MLX-2bit
• majentik/Qwen3.6-27B-MLX-3bit (this repo)
• majentik/Qwen3.6-27B-MLX-4bit
• majentik/Qwen3.6-27B-MLX-5bit
• majentik/Qwen3.6-27B-MLX-6bit
• majentik/Qwen3.6-27B-MLX-8bit
• majentik/Qwen3.6-27B-MLX-MXFP4
• majentik/Qwen3.6-27B-MLX-NVFP4

Provenance

• Upstream: Qwen/Qwen3.6-27B @ 6a9e13bd6fc8f0983b9b99948120bc37f49c13e9
• Quantization: bits=3, mode=affine, group_size=64 (text tower only)
• Toolchain: mlx 0.31.2, huggingface_hub 0.36.2
• Full details in PROVENANCE.md in this repo.

License + attribution

Quantized by majentik from Qwen/Qwen3.6-27B. All rights in the original model remain with its authors.

The upstream model is released under the Apache License 2.0 — see the upstream LICENSE file.