qwen36-35b-a3b-stq1_0

Qwen3.6-35B-A3B — STQ1_0 Ternary (Path 1)

A ternary (STQ1_0, 1.31 bpw, 3:4 sparsity) mixed-precision GGUF of Qwen/Qwen3.6-35B-A3B, built by crushing the routed experts on the most salient layers and keeping the parts that shouldn't be crushed at higher precision.

This is the MIX-STQ1_0 recipe from Tencent's Hy4-preview, moved onto Qwen3.6-35B-A3B (35B MoE, 256 experts, hybrid Gated DeltaNet + Gated Attention). Same idea: most of the expert mass goes ternary, the recurrent state and router stay sharp.

Note on the auto-detected tag: Hugging Face reads the filename and labels this TQ1_0. That is misleading. This is a mixed-precision GGUF, not a single TQ1_0 quant. The actual scheme is below.

Quantization map

Result: 2.69 bpw, 11.4 GiB.

What's in the box

• qwen36-stq1_0.gguf — 11.4 GiB, 2.69 bpw
• qwen36-stq1_0.tensortypes — the per-tensor recipe (read it, it's short)

The recipe, in plain terms

• Routed experts: ternary STQ1_0 on the 29 highest-importance layers (picked from bartowski's imatrix), IQ2_XXS on the other 11
• Down-projection experts: IQ3_XXS (they write to the residual directly, so they get a bit more)
• Gated DeltaNet SSM state (ssm_a / ssm_alpha / ssm_beta / ssm_out): Q8_0, never ternary. The recurrent state is the part you don't want to wreck, and ternary on it quietly destroys long-context recall
• MTP head (nextn.*): Q4_K. bartowski's imatrix skips it, so it's pinned by hand
• Router, embeddings, attention: Q8_0 / Q4_K / Q5_K as appropriate
• output.weight left at bf16

Important: this needs CUDA

STQ1_0's MoE kernels are CUDA-only. The Metal backend in llama.cpp has no STQ1_0 mul_mat_id path, so this file will crash on Apple Silicon. To run it you need a custom llama.cpp build from the pr-22836 branch (the one that adds STQ1_0) with GGML_CUDA=ON. Any CUDA card works — an RTX 3080 with partial offload is fine if you've got the RAM for the rest, though 8 GB VRAM means most layers run on CPU.

llama-cli -m qwen36-stq1_0.gguf -ngl 20 -p "..."

Honest caveats

• I could not measure perplexity for this file on the machine I built it on (no CUDA there). The quality story rests on the architecture: the SSM state is protected, which is the main risk. A real quality number needs a CUDA box.
• 25 GiB is a lot to ship; this repo carries the weight plus the recipe so you can reproduce or adjust the split.

Built as part of a two-path experiment; see the sibling repo qwen36-35b-a3b-p2-iq-mix for an Apple-Silicon-runnable version.