Qwen3.6-27B-GGUF

<div style="border:2px solid currentColor; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,monospace;"> <div style="border-bottom:1px solid currentColor; padding:6px 12px; font-size:11px; letter-spacing:3px; text-transform:uppercase; opacity:0.7; text-align:center;">KIKOCIS // QWEN3.6-27B GGUF // MEASURED, NOT ASSERTED</div> <div style="padding:14px; display:flex; flex-wrap:wrap; align-items:center; justify-content:center; gap:18px;"> <pre style="margin:0; font-size:9px; line-height:1.15;"> BF16 reference quant ┌───────────┐ ┌───────────┐ │ ░░░░░░░░░ │──KLD──▶│ ▓▓▓▓▓▓▓▓▓ │ │ ░░░░░░░░░ │ │ ▓▓▓▓▓▓▓▓▓ │ └───────────┘ └───────────┘ │ │ └──── same top-1? ────┘ 94.29% / 93.29% / 89.32% </pre> <div style="text-align:center;"> <div style="font-size:23px; font-weight:800; letter-spacing:1px;">Qwen3.6-27B-GGUF</div> <div style="font-size:12.5px; letter-spacing:1px; opacity:0.8; margin-top:5px;"><span style="white-space:nowrap;">hybrid SSM+attention</span> · <span style="white-space:nowrap;">27.32B</span> · <span style="white-space:nowrap;">262K native ctx</span> · <span style="white-space:nowrap;">KLD-measured</span></div> </div> </div> <table style="display:table; table-layout:fixed; width:100%; margin:0; border-collapse:collapse; font-size:12px;"> <tr> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">FORMAT</div><div style="font-weight:700;">GGUF</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">SIZES</div><div style="font-weight:700;">13.5–16.8 GB</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">ARCH</div><div style="font-weight:700;">qwen35</div></td> <td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">CONTEXT</div><div style="font-weight:700;">262,144</div></td> </tr> <tr> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">IMATRIX</div><div style="font-weight:700;">yes, 277 chunks</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">VALIDATION</div><div style="font-weight:700;">KLD + Top-1</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">RUNS ON</div><div style="font-weight:700;">llama.cpp</div></td> <td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LICENSE</div><div style="font-weight:700;">Apache-2.0</div></td> </tr> </table> </div>

Qwen3.6-27B — GGUF quants with a fidelity table

There are already many GGUF builds of this model, several of them excellent and far more downloaded than this one will ever be. This repo exists for one reason: it publishes what the others don't — how far each quant's output distribution actually drifts from the unquantized model, measured with KL divergence and top-1 agreement, not asserted.

And that measurement turned up something worth your attention before you pick a file.

⚠️ Read this before choosing by perplexity

Reference: BF16 GGUF, PPL 5.963.

Q3_K_M has the best perplexity of the three — better than the unquantized model itself. If this card only reported PPL, like most quant repos do, the obvious read would be "take the smallest one, it even beats the original."

That read is wrong. By KL divergence Q3_K_M is the least faithful of the three: its median divergence is 4× Q4_K_M's, and it picks a different most-likely token than BF16 on 11 % of positions versus 5.7 % for Q4_K_M. It isn't better — it's different, and wikitext happens to reward that particular difference. Perplexity measures how comfortable a model is on one corpus; it does not measure whether you still have the model you think you have.

That is the whole argument for KLD, and it is why this table exists.

📈 Charts

The third chart is worth a look: divergence is not spread evenly. The median token is nearly untouched at every tier — it's the tail that carries the damage, and the tail is where reasoning chains and rare tokens live.

✅ Recommended files

📦 Files

Q5_K_M and Q6_K are not here. They were not built — the machine that produced this ladder ran out of storage, and the tiers were built smallest-first on purpose so that the sizes most people download survived the cut. Saying so is cheaper than pretending the ladder is complete.

🧠 What this model actually is (it matters for context)

Qwen3.6-27B is not a plain transformer. Of its 65 blocks:

• 48 are SSM / linear-attention layers (ssm_conv, ssm_scan, gated deltas),
• 16 are full-attention layers — every 4th block,
• 1 is an MTP (multi-token-prediction) head.

Only those 16 attention layers carry a KV cache that grows with sequence length. That is why a 262,144-token context is tractable here in a way it would not be for a dense 27B, and it is the most interesting property of this model.

About the MTP head: it is present in these files, and llama.cpp explicitly ignores it (unused tensor blk.64.nextn.* — ignoring). It costs you a little disk and does nothing under llama.cpp today. Repos with -MTP- in the name are making the same tensors available for runtimes that do use them.

🚀 How to run it

# llama.cpp
llama-cli -m Qwen3.6-27B-Q4_K_M.gguf -c 32768 -no-cnv -p "your prompt"

# Ollama
ollama create qwen3.6-27b -f Modelfile

-no-cnv matters: without it recent llama-cli builds drop into conversation mode and wait on stdin, which looks exactly like a hang if you are scripting it.

Context Modelfiles for 8K / 32K / 128K / 262K are in configs/, with the memory trade-off written out in CONTEXT.md.

🔬 Evaluation methodology

Reproducibility is the point of this repo, so here is exactly how every number above was produced.

• Reference: BF16 GGUF converted from the original safetensors, 866 tensors, verified complete by walking every tensor offset to the end of the file.
• Eval corpus: wikitext-2 test split — deliberately disjoint from the imatrix calibration corpus (source code + general prose). Measuring on your calibration data inflates the result.
• Sample size: 8 chunks × 2048 tokens = 16,384 tokens. This is small, and it is small for a concrete reason: this model's vocabulary is 248,320 tokens, so llama.cpp's reference-logit file costs ~497 KB per token. The direction of the Q3_K_M effect is solid (mean ln(PPL(Q)/PPL(base)) = −0.062 ± 0.011), but treat the magnitudes as indicative, not tight.
• Metrics: llama-perplexity --kl-divergence --kl-divergence-base, giving KLD mean/median/ percentiles, Δp statistics and top-1 agreement ("same top p"). Per-quant logs in reports/.
• imatrix: 277 chunks over a ~513 KB corpus, 992 entries covering blocks 0–63. Block 64 (the MTP head) has no coverage because llama.cpp does not use it.
• Quantization source: every tier was quantized from the BF16, never from another quant — deriving a tier from a tier compounds error and would make this table measure the wrong thing.
• Max KLD is ~27 for all three tiers. That is a handful of rare tokens where any quantization falls apart; read the median and p95, not the max.

What is NOT measured here

• No agentic/SWE benchmark was run. This repo makes no claim about tool use or coding ability.
• The 262,144 context is the model's declared native window, not a verified one — a needle-in-a-haystack run at 128K/256K is pending. Do not read "262K" here as "tested at 262K".
• No throughput numbers, by policy.

🔁 Provenance

• Scripts: scripts/ — convert, imatrix, ladder, metrics. The exact ones used.
• Checksums: reports/artifact-sha256sums.txt — verify with shasum -a 256 -c.
• imatrix format: written as GGUF (current llama.cpp default). Older builds need --output-format dat.
• Weights: unmodified. This is a faithful quantization, not a fine-tune or a merge.

🙏 Credit & alternatives

Model, weights and training are © Qwen — Qwen/Qwen3.6-27B, Apache-2.0. All this repo adds is the quant ladder, the fidelity measurements and the context configs.

Other GGUF builds of this model, offered honestly because several are better maintained and more complete than this one: unsloth, bartowski, lmstudio-community, ggml-org. If you want the full ladder or the MTP variants, go there. Come here for the numbers.

🗒️ Changelog

• 2026-08-14 — first release: Q3_K_M, IQ4_XS, Q4_K_M with imatrix, KLD/PPL/top-1 table, charts, scripts and checksums. Q5_K_M/Q6_K deferred (storage). Needle test and agentic probe pending.