Qwen3.8-27B-Hi-Fi-GGUF

<img src="https://huggingface.co/fraQtl/Qwen3.8-27B-Hi-Fi-GGUF/resolve/main/Qwen3.8-27B-Hi-Fi-GGUF.png" alt="Qwen3.8-27B-Hi-Fi-GGUF">

Qwen3.8-27B Q4_K_M GGUF: fraQtl Hi-Fi (calibration-aware quantization)

Q4_K_M at 16,464,274,816 bytes (unsloth UD-Q4_K_M: 16,464,440,224, within 0.001%): KLD to the Q8_0 teacher 0.12297 code/math and 0.23120 general, against 0.22493 and 0.28209 for the canonical build (−45.3%, −18.0%).

Text-only GGUF (the vision tower is not converted). Method: fraQtl calibration-aware per-tensor quantization of Qwen/Qwen3.8-27B, byte-matched to the leading community Q4_K_M.

Fidelity to the original model

Metric: symmetric top-20 KLD on the support of a Q8_0 teacher quantized from the original pinned bf16 checkpoint. 3 independent runs, run-to-run drift ≤1.4e-17 (byte-deterministic rig), one shared teacher and one pinned eval slice for every arm.

Margins are vs the canonical arm. This build is byte-matched to the unsloth canonical by construction (Δ165,408 B, 0.001%); the ggml-org file is a different, larger build (18.97 GB, no NextN block). Each row is fidelity at its stated size.

Wikitext-2 perplexity is +0.7% for this build vs canonical, and general-slice top-1 token agreement is −0.2pp; the canonical is lower on perplexity and higher on top-1. Both are in the build receipt.

Task benchmarks

GSM8K (n=200) and MATH-500 (n=500), 0-shot instructed-CoT, identical harness for all three arms:

95% confidence interval: GSM8K ±6.9pp at n=200; MATH-500 ±4.4pp at n=500. Every pairwise difference is inside it. Per-item paired GSM8K audit, ours vs canonical: 4/200 items differ, a 3–1 split, exact two-sided sign test p = 0.625.

Long-context retrieval (needle-in-a-haystack)

Grid: contexts {8K, 32K} × depths × 3 passkeys, plus 131K cells × 2 passkeys, for 63 cells per arm per run, multi-run. All three arms pass every cell of every run (270 individual passkey checks total).

Scope

• Task benches are n=200 (GSM8K) and n=500 (MATH-500).
• Calibration: text imatrix from the A1-selector winner mix_cm70_gen30 at ctx 8192; the vision tower is not converted.
• 262K is this model's native context; the numbers on this card are a quality-tier GGUF measurement, not a KV/serving measurement.

Methods note

• One MATH-500 cell (ours) ran twice after a preemption that looked like a wedge; the re-fire and the original delayed run both completed and returned the same score (438/500). Detail in the taskbench receipt.

Provenance

Receipts for every number are in receipts/ (JSON: pins, conversion gate, calibration selection, build + 3-run KLD, NIAH grid, task benches).

Credit

• Discussion Qwen/Qwen3.8-27B#65 (d4nnyy/AtomicChat): prior day-0 stored-logits KLD across 36 quants (4096-ctx, no retrieval gate), credited as prior art. Our scope differs: independent teacher, 32K–131K contexts, needle-gated, 3-run.
• unsloth: the canonical community build (4.3M downloads/30d), UD-Q4_K_M class.
• ggml-org / llama.cpp: toolchain and second-arm GGUF.

Files

The fraQtl ladder

More from fraQtl

The serving lane (KV-cache compression sidecars for vLLM) holds nine concurrent ≈128K-context users on a single A100 (134.1 tok/s aggregate, 9/9 per-user retrieval checks, receipt 2026-08-14): fraQtl/qwen3-4b-instruct-2507-kv-sidecars. Org page: huggingface.co/fraQtl.