Qwen3.5-4B-QD-GGUF

Qwen3.5-4B — QD (Quadux Dynamic) GGUF

Measured per-tensor quantization maps for Qwen/Qwen3.5-4B, built from unsloth's MTP conversion (MTP head and vision projector included in every file). At 2–4 GB every percent of quality matters — this is where measured allocation pays the most.

Method (probes → error matrix → knapsack → measured gradient/hillclimb) and honesty rules are identical to our Qwen3.6-35B-A3B-QD release — see its BENCHMARKS.md for the full methodology record.

Benchmarks — all comparisons at byte parity, same toolchain

Mean KL divergence vs BF16 logits (60×512 tokens; en = wikitext-2-test, de = German Wikipedia sample). "UD map, our build" re-quantizes unsloth's own type map with the identical llama.cpp + imatrix — allocation against allocation, no toolchain advantage.

KL divergence measures how far the quantized model's output distribution drifts from the full-precision (BF16) model — lower is better, 0 = identical to BF16. Every number was measured on one rig (RTX 5090, CUDA), against the same BF16 logits, on the exact files published here.

Each QD file is the same size as — in fact slightly smaller than — the unsloth file it is compared against, so this is a like-for-like comparison at equal size: same bytes, less divergence.

† Q6 uses unsloth's own allocation as the starting point (their file 0.001931, rebuilt with our toolchain), then one measured hillclimb move; see below.

Rebuilding unsloth's own maps with our toolchain changes their numbers by <1.5% (e.g. Q4: 0.017401 vs their file 0.017662) — on this dense model there is no toolchain advantage; the gains above are pure allocation. German gains exceed English throughout: the usual calibration is English-leaning, ours optimizes the mean of both.

Q5_K_XL: use unsloth's. Our best attempt measured 0.00948 vs their 0.00892 on equal footing; we link rather than ship a worse file. The Q6 rung above was found by exhaustive measured local search (34 real-KLD swap moves from their map; 1 accepted, 33 rejected — all documented).

Larger quants (Q5, Q8, BF16): take them straight from the source

At Q8 and above the quantization damage is already negligible (uniform Q8_0 = 0.00152 KLD, vs 0.0131 at Q4), so a "QD-Q8" would be a label on a file practically identical to a standard Q8_0. We link rather than duplicate:

QD earns its keep below Q5, and most of all at Q2/Q3 — that is where measured per-tensor allocation beats a fixed ladder by the widest margin.

Notes

• Uniform anchors (en): Q8_0 0.00152 · Q6_K 0.00418 · Q5_K 0.01513 · Q4_K 0.03860 · Q3_K 0.16889. Note our Q3_K_XL (0.0292) beats uniform Q4_K (0.0386): Q4-class quality at Q3 size.
• Per-tensor maps in tensortypes/ — rebuild or audit any file.
• imatrix: unsloth's published one (Apache-2.0, credit to them). Base model © Alibaba Cloud, Apache-2.0.
• Serving: standard llama.cpp; --mmproj mmproj-F16.gguf for vision; MTP speculation via --spec-type draft-mtp.

Deutsche Kurzfassung

Gemessene statt regelbasierte Typkarten. Bei Byte-Parität und identischem Werkzeug: −21 % (Q4), −48 % (Q3), −56 % (Q2) KL-Divergenz gegenüber unsloths Dynamic-Karten; auf Deutsch fällt der Abstand noch größer aus. Bei Q5 ist unsloths Karte besser — wir verlinken sie, statt eine schlechtere eigene Datei anzubieten. Q6 stammt aus erschöpfender messbasierter Lokalsuche (34 Züge, 1 angenommen). Quantisiert von Quadux IT.

License

This model and its base model are licensed under the Apache License 2.0, which permits commercial and research use, modification, and redistribution, subject to the standard requirements: include the copyright notice, the license text, and a NOTICE of any changes.

• Base model: Qwen/Qwen3.5-4B — Apache 2.0, © Alibaba Cloud.
• This repository: Apache 2.0 (same terms as the base model).
• Modifications by Quadux IT GmbH: GGUF quantization only — QD (Quadux Dynamic) per-tensor type maps, built on unsloth's published imatrix (Apache 2.0) and MTP conversion. No change to the base weights or architecture.

If you redistribute these files, include the Apache 2.0 license text and attribution to the upstream Qwen team, to unsloth (imatrix + MTP conversion), and to Quadux IT GmbH. See NOTICE and LICENSE.

Citation

The original Qwen3 work — cite this if you publish results using the model:

@misc{qwen3.5,
  title  = {Qwen3.5},
  author = {Qwen Team},
  year   = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Qwen/Qwen3.5-4B}}
}

To cite this quantization specifically:

@misc{quadux_qd_qwen3_5_4b,
  author       = {{Quadux IT GmbH}},
  title        = {Qwen3.5-4B-QD-GGUF (Quadux Dynamic Quantization)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/QuaduxIT/Qwen3.5-4B-QD-GGUF}},
  note         = {Measured per-tensor GGUF quantization of Qwen/Qwen3.5-4B; built on unsloth's imatrix and MTP conversion}
}

---

About Quadux IT GmbH

Software for engineering offices and accounting pipelines. Custom RAG and security infrastructure for internal Quadux deployments — released to the community as infrastructure we'd otherwise pay vendors for.

Find more at quadux.it · contact [email protected]

Imprint

Quadux IT GmbH · Schulstr. 3 · 37139 Adelebsen · Germany Registered Göttingen, HRB 206773 · VAT ID DE353975332 · DUNS 344198559 Managing Director: Walter Hoffmann