Qwen3.8-27B-AP-GGUF

Qwen3.8-27B · Agention Precision GGUF

Same size. Same speed. Closer to the model Qwen trained than any other quant

Full Results

Agention Precision quants are the highest precision quants of Qwen3.8 27B byte-for-byte. Each file has a non-uniform assignment and low-loss error correction built with a custom encoder.

This is a drop-in GGUF pack for Qwen3.8-27B: standard llama.cpp types, no fork, no flags. Vision projector and MTP draft head included.

Every 27B quant gives something up. These give up less. We measured the leading public GGUFs against the full BF16 model on one protocol. At every size we ship, ours is the closest to Qwen’s next-token distribution.

Every quant is tested against three corpora: unseen technical prose, general web text and Wikipedia.

Swap cost is zero. VRAM and tokens/sec stay the same. The file behaves more like the weights Qwen released.

VRAM = weights + q8_0 KV + llama.cpp buffers. Only a quarter of layers use full attention, so 32k context is about 1 GiB at q8_0.

Take AP-IQ4_XS if it fits. Step down only for memory.

---

Measured

Fidelity = KL divergence of the next-token distribution vs Qwen3.8-27B BF16. Lower is closer to the original.

60 × 2048 tokens, three corpora, same build, same BF16 logits:

• held-out — 1.85 MB of our own technical prose, never part of any calibration set. It is internal engineering documentation, so it is not published: this column is the one you cannot re-run yourself.
• neutral web — mixedweb-v1, 800,789 chars, a seeded FineWeb slice, md5 51e0045e8cabf37922aa82766a25b7b4, published with its per-document manifest and builder in agentionai/quant-fidelity-corpora
• wikitext-2 — the standard wiki.test.raw, 1,288,556 chars, md5 7c0137fc034ddbc56a296bce31b4f7fb (llama.cpp/scripts/get-wikitext-2.sh)

Reproducing these numbers

The reference is Qwen/Qwen3.8-27B at revision 1d4bf0f2 (the weights uploaded 2026-08-13; later commits are README only), converted with llama.cpp's own converter — no changes, no fork:

python convert_hf_to_gguf.py --outtype bf16 Qwen3.8-27B/ --outfile Qwen3.8-27B-BF16.gguf   # 54,657,733,888 bytes

Then one base of BF16 logits per corpus, and every quant scored against it. The KLD code is upstream and unmodified; ours was built at llama.cpp 26bc85e42:

llama-perplexity -m Qwen3.8-27B-BF16.gguf -f mixedweb-v1.txt \
  --kl-divergence-base base-mixedweb.bin -c 2048 --chunks 60 -ngl 99

llama-perplexity -m Qwen3.8-27B-AP-IQ4_XS.gguf --kl-divergence \
  --kl-divergence-base base-mixedweb.bin -c 2048 --chunks 60 -ngl 99

Mean KLD and Same top p in that output are the two numbers in the tables above; 99.0% KLD is the worst-1 % column. Every file on this page — ours, Unsloth's, ISTA-DASLab's, AtomicChat's — was measured with those exact commands, same build, same base files, same 60 × 2048 tokens.

Head to head with Unsloth, identical bytes

Same tensor types, same file size, same speed, same memory. What differs is how the weights inside each block were chosen — a different calibration and a different encoder.

Held-out gains: 3.2σ, 3.2σ, 5.7σ, 16σ and 18σ (Q4_K_XL / Q4_K_M / IQ4_XS / Q3_K_XL / IQ3_S). Neutral-web gains: 3.7σ at Q4_K_XL, 3.4σ at Q3_K_XL, 3.1σ at IQ3_S; 2.4σ at Q4_K_M. Wikitext-2 is a statistical tie (under 1.5σ) at every tier except IQ3_S, where unsloth is 7% closer (2.1σ).

Top-1 match with BF16 on held-out text: 92.8% · 92.2% · 90.4% · 90.1% · 88.6% · 87.7% · 85.8% (Q4_K_XL / Q4_K_M / IQ4_XS / Q3_K_XL / IQ3_S / IQ3_XS / IQ3_XXS).

The field near these sizes

ISTA rows are their -mtp builds (draft head included, same as ours).

† AP-IQ2_S (2026-09-28) and AP-IQ3_XXS, AP-IQ3_XS, AP-IQ3_S and AP-Q3_K_XL (2026-09-29) were rebuilt with an extended calibration set that shares 1.4% of its 12-grams with the held-out corpus. Their held-out figures are therefore measured on the 95% of held-out text with zero overlap. That subset is slightly harder: every file we scored on both reads about 3% higher on it than on the full set (ISTA GSQ-RCO-IQ2_S 0.1541 → 0.1584, the previous AP-IQ2_S 0.1418 → 0.1460), so comparing it with the other rows' full held-out figures is conservative. The extended calibration moves precision toward technical and code text: against the previous files, held-out KL drops 15–18% while wikitext-2 reads up to 9% higher.

KL is fidelity to Qwen’s predictions, not a task leaderboard. The claim it supports is narrow and checkable: at every size we ship, you are closer to the original than the same-size alternative.

First downstream check. MMLU-Pro, 70 questions (5 per subject), thinking mode with Qwen’s recommended sampling, one run, same questions for both files:

---

Running

Use the sampling settings from the Qwen3.8-27B model card. Thinking is on by default. To turn it off per request, send "chat_template_kwargs": {"enable_thinking": false}.

llama-server -hf agentionai/Qwen3.8-27B-AP-GGUF:IQ4_XS \
  --jinja -ngl 999 -fa on -c 32768 -ctk q8_0 -ctv q8_0

Keep the KV cache at q8_0 or f16. A 4-bit value cache makes long reasoning traces degenerate into repetition on this model family.

LM Studio: search for agentionai/Qwen3.8-27B-AP-GGUF and pick a tier. Ollama: ollama run hf.co/agentionai/Qwen3.8-27B-AP-GGUF:IQ4_XS

llama-server -hf agentionai/Qwen3.8-27B-AP-GGUF:IQ4_XS \
  --mmproj mmproj-BF16.gguf --jinja -ngl 999 -fa on -c 32768 -ctk q8_0 -ctv q8_0

Built with

The tiers are built and verified with our own Rust tooling agention-infer. Every tier is measured against BF16 on all three corpora, and shipped only if it beats the same-size alternative. unsloth's dynamic type maps underpin AP-Q4_K_XL, AP-Q4_K_M, AP-IQ4_XS, AP-Q3_K_XL and AP-IQ3_S, and we thank them for that work. AP-IQ3_XS and AP-IQ3_XXS sit at sizes no one else ships and use our own per-tensor allocation, so they are compared against the nearest published files above and below them. AP-IQ2_S is built on ISTA-DASLab's GSQ-RCO type map for that size, re-encoded with our calibration and tooling, and we thank them for that work. imatrix-mixed-v2.gguf, the importance matrix from our calibration pass, is included for anyone building their own quants. Since 2026-09-28 AP-IQ2_S, and since 2026-09-29 AP-IQ3_XXS, AP-IQ3_XS, AP-IQ3_S and AP-Q3_K_XL, use an extended calibration set that adds agentic coding transcripts; its imatrix is not published.

Calibration is ordinary public text: the Bartowski and Thireus imatrix corpora, plus a one-third share of whole articles from the wikitext-2 train split — never the test split, and the 12-gram overlap with each of the three evaluation corpora above is 0, 0 and 6 (section headings and unit conversions). 6,012 documents, 4.67 M characters, md5 b8e3269f085f500ab307b0cc977126b0, 1,200 × 512 tokens through the BF16 model. The calibration is not the advantage here; anyone can build on the same text.

Does it hold up in real use? A one-shot coding test

Same prompt, same five seeds for every file, first answer only, no retries and no fixes. Each page opened in Chrome and judged by eye.

Five trials per file is a small sample; read it as a sanity check that the lower KL carries over, not as a benchmark.

More tiers follow as they clear the same bar.

Credits and license

Built and tested on AMD Ryzen AI Max (Strix Halo). Hardware supported by AMD

Base model: Qwen/Qwen3.8-27B (Qwen community license 1.0, see LICENSE).

Support AgentionAI

These quants are released freely. If they save you VRAM or make Qwen more useful, you can buy me a coffee or some GPU time and sponsor continued quantization and benchmarking on Buy Me a Coffee. AgentionAI is a one-person team and can use your help.