Qwen3.6-27B-UD-Q4_K_XL-AWQ-MTP-mlx

Qwen3.6-27B — UD-Q4_K_XL + AWQ + MTP (MLX)

Mixed-precision 4-bit quantization of Qwen/Qwen3.6-27B for Apple Silicon, built with mlx-node using the Unsloth Dynamic class map plus AWQ pre-scaling from an activation imatrix.

The multi-token-prediction (MTP) head is preserved. All 15 mtp.* tensors are kept inline in the main shards, unquantized in BF16, so a runtime that supports Qwen3.5/3.6 speculative decoding can use them.

This is the one deliberate difference from the Brooooooklyn UD-Q*_K_XL builds this recipe otherwise follows — I checked their Q4 checkpoint and it carries zero mtp.* tensors. Other MLX conversions of this model do ship MTP; this is not a claim to be the only one.

Contents

affine quantization, group size 64, 370 per-tensor overrides. 27,781,427,952 total parameters.

Effective bits per weight

6.18 BPW — 20,889,186,434 bytes on disk over 27,020,391,152 logical parameters.

This is well above the nominal 4, and that is expected rather than a defect. --q-bits 4 sets the base class in the Unsloth Dynamic map, not the average. Only gate_proj and up_proj actually sit at 4-bit; attention projections, the GatedDeltaNet input projections, down_proj, the embeddings and lm_head are all promoted. The BF16 vision tower and BF16 MTP head add further weight that no quantizer touches. Any "Q4" MLX quant of this family that reports a similar size is doing the same thing.

Precision map

Left in BF16 throughout: all RMSNorms, q_norm/k_norm, the GatedDeltaNet A_log / conv1d / dt_bias state parameters, the entire vision tower, and the entire MTP head.

AWQ calibration

AWQ pre-scaling amplifies activation-important weight columns and folds the inverse into the preceding norm — an output-preserving reparametrization that moves quantization error onto channels that matter less. Applied here across four dependency groups: norm→gate/up, up-rows→down-cols, input_layernorm→q/k/v, and input_layernorm→GatedDeltaNet in_proj_*.

The importance matrix is Unsloth's own calibration for this exact model — imatrix_unsloth.gguf_file from unsloth/Qwen3.6-27B-GGUF, internally tagged unsloth_calibration_Qwen3.6-27B.txt, 992 tensors over 76 chunks. An imatrix from a different checkpoint would apply without error and silently degrade the result, so this provenance matters.

Reproducing

mlx convert \
  --input  <Qwen/Qwen3.6-27B snapshot> \
  --output Qwen3.6-27B-UD-Q4_K_XL-AWQ-MTP-mlx \
  --model-type qwen3_5 \
  --quantize \
  --q-bits 4 \
  --q-group-size 64 \
  --q-recipe unsloth \
  --imatrix-path imatrix_unsloth.gguf_file \
  --q-mtp off

--q-mtp off means "do not split the MTP head into a separate drafter directory" — the tensors stay inline and unquantized. Use --q-mtp split instead if you want a standalone drafter.

Conversion cost on a 36 GB M-series Mac: 99 s, peak 15.85 GiB process RSS, 22.7 GiB MLX allocator peak. Swap: 7.89 GiB paged out / 5.76 GiB paged in (vm_stat, sampled across the run).

This needs a build of mlx-node containing the bounded-memory conversion fix (PR #118); before it, the AWQ path materialized the whole BF16 checkpoint at once and would not complete in 36 GB.

Variants

Sources

• Base weights — Qwen/Qwen3.6-27B
• Importance matrix — unsloth/Qwen3.6-27B-GGUF
• Quantization strategy — Unsloth Dynamic
• Conversion tool — mlx-node
• Recipe reference — Brooooooklyn/Qwen3.6-27B-UD-Q4_K_XL-mlx, whose flag set this follows apart from MTP retention

Not tested

Stated plainly so you can weigh it:

• No benchmarks were run on this build. No perplexity, no task evals, no throughput numbers. Nothing here claims a quality or speed result.
• MTP speculative decoding is preserved, not verified. The tensors are present and correctly shaped; whether your runtime engages them is on your runtime.
• The vision tower is carried through unquantized but untested. No image or video input was exercised.
• License is stated as Apache-2.0 following the upstream Qwen3 convention; verify against the base repo if it matters to you.