Downloads · 30 days
36
6% of all-time downloads
xero0000/Qwen3.6-27B-vram14-GGUF
Qwen3.6-27B-vram14-GGUF is a text generation model from xero0000. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
A dense 27B that fits entirely in 18 GB of VRAM, keeps its native multi-token-prediction head, and measures statistically lossless against its Q8 source.
Downloads · 30 days
36
6% of all-time downloads
All-time downloads
642
Public
Repo size
14.1 GB
Likes
1
Public
Click a slice to open those files.
.gguf14.1 GB · 100%
From the Hugging Face model README
A dense 27B that fits entirely in 18 GB of VRAM, keeps its native multi-token-prediction head, and measures statistically lossless against its Q8 source.
Qwen3.6-27B ships with a trained MTP head (nextn_predict_layers=1) that most quants throw away. This build keeps it: with -mtp in ik_llama.cpp you get lossless self-speculative decoding — +58 % decode on code in our tests, from a head that was actually trained for these weights.
| File | Size | ctx |
|---|---|---|
Qwen3.6-27B-vram14-imat.gguf | 14.1 GB | 64 K+ all-VRAM on 18 GB (more with spare VRAM) |
| Quant | Size | PPL |
|---|---|---|
| Q8_K_XL (source) | 35.8 GB | 2.5207 ± 0.073 |
| vram14 (this repo) | 14.1 GB | 2.4955 ± 0.069 |
The quant scores at its source's ceiling (inside the error bars) at 39 % of the size — the imatrix + selective-precision recipe is effectively lossless here.
| decode | prefill @14.5 K | |
|---|---|---|
code (with -mtp) | 52 t/s | 670 t/s |
| prose | 33 t/s | — |
Dense models read every weight per token, so decode scales directly with VRAM bandwidth — a single 3090/4090-class card will be substantially faster than this dual-mid-range split.
Requantized from Unsloth's UD-Q8_K_XL with an imatrix (61 K calibration tokens, mixed code/prose):
ffn_up / ffn_gate → IQ3_S (the size lever; imatrix-guided)Built with ik_llama.cpp llama-quantize (--ffn-up-type iq3_s --ffn-gate-type iq3_s --output-tensor-type q6_K --token-embedding-type q6_K).
ik_llama.cpp required for -mtp; the file loads in mainline llama.cpp without it (head ignored).
llama-server -m Qwen3.6-27B-vram14-imat.gguf \
--jinja -fa on --cache-type-k q4_0 --cache-type-v q4_0 \
--ctx-size 65536 -ngl 99 --tensor-split 46,54 --ubatch-size 256 --no-mmap \
-mtp --ctx-size-draft 8192
Single ≥16 GB GPU: drop --tensor-split. Qwen3.6 is a reasoning model — thinking goes to reasoning_content; disable with "chat_template_kwargs": {"enable_thinking": false}.
On the same held-out corpus, Qwen3.6-35B-A3B (MoE) quantized to a similar all-VRAM size scored better (2.43) and decoded 3× faster — MoE wins when total VRAM is the constraint. This dense build is for when you specifically want dense-model behavior (no routing variance, stronger per-token compute) or have the bandwidth to feed it.
Built on a Ryzen 5950X + 3060 Ti + 3080. Base model © Alibaba Cloud, Apache-2.0. Quantized with ik_llama.cpp tooling.