Qwen3.6-27B-MLX-6bit

Qwen3.6-27B-MLX-6bit

A 6-bit quantization of Qwen/Qwen3.6-27B for Apple Silicon (MLX). In our evaluation this is the low-degradation sweet spot: statistically indistinguishable from the BF16 model while being ~2.6× smaller and faster.

• 6.50 bits/weight, ~21 GB on disk
• ~25 tok/s decode, ~22 GB peak memory on an Apple M5 Max
• Weight-only affine quant (activations stay FP16), group_size=64
• Effectively lossless (see measurements)

Want smaller/faster and can accept a measurable quality cost? See the mixed-4bit build (17 GB, ~30 tok/s). Want maximum fidelity? See the 8-bit build.

Measured quality (vs BF16)

Perplexity, KL-divergence, and top-1 agreement against the BF16 model on a fixed 650-token multi-domain passage. Indicative smoke test, not a full benchmark — a relative comparison between builds.

¹ within noise — 6-bit is statistically indistinguishable from BF16 on this sample. There is a sharp quality "cliff" below 6-bit: dropping to 4-bit costs ~+5% perplexity and ~6 points of top-1 agreement.

Usage

pip install mlx-lm
mlx_lm.generate --model EigenLabs/Qwen3.6-27B-MLX-6bit \
  --prompt "Explain gated delta networks in two sentences." --max-tokens 256

from mlx_lm import load, generate
model, tokenizer = load("EigenLabs/Qwen3.6-27B-MLX-6bit")
messages = [{"role": "user", "content": "What is 17*23?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

Qwen3.6 is a reasoning model — the chat template opens a <think> block, so responses begin with a chain of thought.

About the model

Qwen3.6-27B is a hybrid architecture: 48 of 64 layers use Gated-DeltaNet linear attention and 16 use full softmax attention, with a 262K context window. At 6-bit, quantization error is negligible across all layer types, so this build uses a uniform 6-bit affine scheme (group_size=64) rather than mixed precision.

Requirements & notes

• Apple Silicon with MLX; ~24 GB free unified memory. Comfortable on 32 GB+, ideal on 48 GB+ for long context.
• Text-only. The MLX qwen3_5 implementation does not load the model's MTP head or its vision tower (use mlx-vlm separately for images).
• Hybrid Gated-DeltaNet layers run on the Metal GPU.

How it was made

from mlx_lm import convert
convert("Qwen/Qwen3.6-27B", mlx_path="Qwen3.6-27B-MLX-6bit", quantize=True, q_bits=6, q_group_size=64)

Tokenizer, chat template, and generation config are copied verbatim from the base model.

License & attribution

Apache 2.0, inherited from Qwen/Qwen3.6-27B (© Alibaba / Qwen team). This is a quantized derivative produced by EigenLabs. Please cite the original Qwen3.6 model.