qwen36-35b-a3b-p2-iq-mix

Qwen3.6-35B-A3B — I-quant Mix (Path 2)

A mixed-precision I-quant GGUF of Qwen/Qwen3.6-35B-A3B that actually runs on Apple Silicon. It's the same protection scheme as the ternary Path 1 build, but the experts use IQ2_XXS / IQ3_XXS instead of STQ1_0, because those have Metal mul_mat_id kernels and ternary doesn't.

This is the version to grab if you're on a Mac. Path 1 (ternary) is CUDA-only and will crash on Metal; this one runs there at a usable speed.

Note on naming: this is a mixed-precision GGUF (IQ2_XXS + IQ3_XXS experts, not a single quant type). The diagram and table below show the exact split.

Quantization map

Result: 3.26 bpw, 13.5 GiB.

What's in the box

• qwen36-p2-iq-mix.gguf — 13.5 GiB, 3.26 bpw
• qwen36-p2-iq-mix.tensortypes — the per-tensor recipe
• eval_ppl_corpus.txt — the held-out text used for the perplexity number below

The recipe

• Routed experts: IQ3_XXS on the 29 highest-importance layers (from bartowski's imatrix), IQ2_XXS on the other 11
• Down-projection experts: IQ3_XXS
• Gated DeltaNet SSM state (ssm_a / ssm_alpha / ssm_beta / ssm_out): Q8_0. Same as Path 1, the recurrent state stays sharp
• MTP head (nextn.*): Q4_K, pinned by hand (bartowski's imatrix skips it)
• Router, embeddings, attention: Q8_0 / Q4_K / Q5_K
• output.weight left at bf16

Run it

Plain llama.cpp with Metal, no custom build needed:

llama-cli -m qwen36-p2-iq-mix.gguf -ngl 99 -p "..."

On an M3 Max it did 491.9 tok/s prompt processing and 70.7 tok/s decode. Unsloth's Mac GGUF inference uses the same Metal backend, so it loads there too.

Quality

Perplexity on a held-out expository corpus (context 512): 5.48 ± 0.53. That's a real number from llama.cpp's own perplexity tool, which handles the hybrid SSM state correctly. A cross-runtime comparison against the MLX oQ4e build isn't clean (different tokenizers, and MLX's perplexity path doesn't thread the Gated DeltaNet state), so take the speed/quality table in the comparison doc as directional, not absolute.

Honest caveats

• This is bigger than Path 1 (13.5 vs 11.4 GiB) because I-quants cost more bits than ternary. You trade a little size for "runs on your Mac."
• Built as part of a two-path experiment; see qwen36-35b-a3b-stq1_0 for the ternary CUDA version.