Downloads · 30 days
338
3% of all-time downloads
zeeksa/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-DFlash-GGUF
Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-DFlash-GGUF is a machine learning model from zeeksa. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Everything needed to run HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive with DFlash speculative decoding in llama.cpp: all target quants, the mmproj (vision), and the DFlash draft head in five quant levels.
Downloads · 30 days
338
3% of all-time downloads
All-time downloads
10.2K
Public
Repo size
179 GB
Likes
4
Public
Click a slice to open those files.
.gguf179 GB · 100%
From the Hugging Face model README
Everything needed to run HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive with DFlash speculative decoding in llama.cpp: all target quants, the mmproj (vision), and the DFlash draft head in five quant levels.
DFlash is lossless — output is byte-identical to running the target alone; the draft only makes generation faster (measured +9–17% over the MTP head, see below). The target and draft are always two separate files loaded together; any target quant pairs with any draft quant.
| File | Size |
|---|---|
| ...-IQ2_M.gguf | 10.0 GB |
| ...-Q2_K_P.gguf | 11.5 GB |
| ...-IQ3_XS.gguf | 12.0 GB |
| ...-IQ3_M.gguf | 12.6 GB |
| ...-Q3_K_P.gguf | 14.3 GB |
| ...-IQ4_XS.gguf | 15.1 GB |
| ...-Q4_K_P.gguf | 17.5 GB |
| ...-Q5_K_P.gguf | 20.8 GB |
| ...-Q6_K_P.gguf | 23.2 GB |
| ...-Q8_K_P.gguf | 32.0 GB |
Plus mmproj-...-f16.gguf (0.93 GB) for vision input.
| File | Size | Notes |
|---|---|---|
| Qwen3.6-27B-DFlash-BF16.gguf | 3.5 GB | Best acceptance — recommended |
| Qwen3.6-27B-DFlash-Q8_0.gguf | 1.9 GB | Near-lossless |
| Qwen3.6-27B-DFlash-Q6_K.gguf | 1.4 GB | |
| Qwen3.6-27B-DFlash-Q5_K.gguf | 1.2 GB | |
| Qwen3.6-27B-DFlash-Q4_K_M.gguf | 1.0 GB | Smallest |
The draft head and target quant are independent choices — llama.cpp loads them as two separate models, so every head in this repo pairs with every target quant above. There is no need for a Q2/Q3/IQ head to match a Q2/Q3/IQ target. Suggested pairing: BF16 head if you have ~3.5 GB VRAM to spare (best acceptance), Q4_K_M head (1 GB) if squeezed. Avoid quantizing the head harder than that: draft quality sets your speedup (we measured code acceptance drop 69% -> 56% just going BF16 -> Q8_0), so a Q2-class head would erase the benefit of speculation.
dflash)llama-server \
-m Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf \
--mmproj mmproj-Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-f16.gguf \
-ngl 99 -fa on \
--spec-type draft-dflash \
--spec-draft-model Qwen3.6-27B-DFlash-BF16.gguf \
--spec-draft-n-max 6 \
--spec-draft-ngl 99
Q8_K_P target + BF16 draft, RTX PRO 6000 Blackwell 96 GB, 200-token greedy completions.
Baseline: same target with the Qwen3.6-27B MTP head (--spec-type draft-mtp, n-max 5).
| Workload | MTP baseline | DFlash | Speedup | DFlash acceptance (mean run len) |
|---|---|---|---|---|
| Code | 108.5 tok/s | 126.6 tok/s | +16.6% | 69.3% (5.10) |
| Math | 146.0 tok/s | 159.9 tok/s | +9.5% | 85.6% (6.03) |
| Chat | 88.4 tok/s | 96.7 tok/s | +9.5% | 43.9% (3.60) |
Q8_0 draft on code: 98 tok/s at 55.6% acceptance — quantizing the draft trades acceptance for VRAM; prefer BF16 unless memory-constrained. The drafter was trained against stock Qwen3.6-27B yet acceptance holds up well on this finetune; output verified identical with and without speculation.
llama-quantize from BF16