Downloads · 30 days
46
1% of all-time downloads
DAXZEIT/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q5_K_XL-gguf
Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q5_K_XL-gguf is a text generation model from DAXZEIT. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Update 20/05/2026 — MTP draft head available: A compatible self-speculative decoding companion is now published: Qwen3.6-27B-Claude-Opus-Reasoning-MTP-Q4KM-gguf — load it via --model-draft for +40% decode throughput a…
Downloads · 30 days
46
1% of all-time downloads
All-time downloads
3.6K
Public
Repo size
80.7 GB
Likes
1
Public
Click a slice to open those files.
.gguf20.3 GB · 100%
From the Hugging Face model README
Update 20/05/2026 — MTP draft head available: A compatible self-speculative decoding companion is now published: Qwen3.6-27B-Claude-Opus-Reasoning-MTP-Q4_K_M-gguf — load it via
--model-draftfor +40% decode throughput at no quality cost (llama.cpp ≥ b9245,--spec-type draft-mtp).
Quantized GGUF of rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled in Unsloth Dynamic 2.0 Q5_K_XL format.
This is the only publicly available UD Q5_K_XL quantization of this model.
⚠️ Chat template fix required — the original rico03 repo ships the Qwen3.5 template, breaking Qwen3.6 tool calls. Add
--chat-template-file qwen3.6_chat_template.jinjato your llama-server command (template source). Tool call format:<function=tool_name>{"param": "value"}</function>
For maximum context (212K on 24GB VRAM), see UD Q4_K_XL. For quality reference, see UD Q6_K_XL.
Part of the DAXZEIT UD SSM-aware series — UD recipes reverse-engineered from Unsloth base models and applied to distills using calibrated imatrix.
Updated 2026-05-12 — If you downloaded this model before 12 May 2026, please re-download. The initial release used an incorrect quantization recipe. This version applies the correct Unsloth Dynamic 2.0 UD recipe with calibrated imatrix.
| File | Size | BPW | Description |
|---|---|---|---|
Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q5_K_XL.gguf | 19 GB | 6.04 | UD Q5_K_XL with imatrix |
Unsloth Dynamic 2.0 is a mixed-precision quantization strategy that selectively upgrades the most accuracy-sensitive tensors above the base quantization level, guided by a per-tensor importance matrix (imatrix). The recipe is reverse-engineered from the official Unsloth base model GGUF and applied to the distill using a calibrated imatrix.
For Q5_K_XL the recipe uses 195 individual --tensor-type overrides on top of a Q5_K_M base:
| Type | Count | Tensors |
|---|---|---|
| Q8_0 | 48 | blk.N.ssm_out.weight — all SSM output projections |
| Q6_K | 176 | output.weight, attn_qkv, ffn_down, attn_v, late-network ffn_gate/up |
| Q5_K | 262 | attn_gate + remaining attention/FFN weights |
| Q4_K | 12 | Mid-network ffn_gate/up (blk.12–16, imatrix-guided) |
| F32 | 353 | Norms, biases, SSM scalars |
The 48 ssm_out.weight tensors (one per block) are kept at Q8_0 — these are the output projections of the SSM recurrent mechanism, critical for long-context coherence in the Qwen3.6 hybrid architecture.
Measured on wikitext-2, full test set (~1M tokens, 1952 chunks, ctx=512), RTX 3090.
| Model | PPL | BPW | Size |
|---|---|---|---|
| Q6_K plain (reference) | 7.4694 ±0.031 | 6.57 | 21 GB |
| UD Q4_K_XL | 7.4712 ±0.031 | 5.41 | 17 GB |
| UD Q5_K_XL (this) | 7.4891 ±0.031 | 6.04 | 19 GB |
| UD Q6_K_XL | 7.4753 ±0.031 | 7.64 | 24 GB |
All four models fall within 0.02 PPL of each other — statistically equivalent (±σ overlap on all). The UD recipe concentrates precision on the tensors that matter most, achieving Q6_K-level quality at a lower bit-per-weight budget.
This model supports vision via a multimodal projector compatible with the Qwen3.6-27B hybrid SSM+Transformer architecture.
mmproj: DAXZEIT/Qwen3.6-27B-mmproj-hybrid-Q8_0-F16-gguf — 601 MB, Q8_0/F16 mixed precision
llama-server \
-m <this-model>.gguf \
--mmproj Qwen3.6-27B-mmproj-hybrid-Q8_0-F16.gguf \
...
llama-server \
-m Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q5_K_XL.gguf \
--mmproj Qwen3.6-27B-mmproj-Q8_0.gguf \
--ctx-size 131072 \
--n-gpu-layers 65 \
--cache-type-k q4_0 \
--cache-type-v q4_0 \
--flash-attn auto \
--port 5000
Fits in 24GB VRAM at 65 GPU layers, Q4_0 KV cache, 131K context with mmproj.
imatrix source: rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled Q6_K — wikitext-2, 200 chunks, ctx 512.
The UD recipe was reverse-engineered from unsloth/Qwen3.6-27B-GGUF UD-Q5_K_XL using GGUFReader tensor extraction. The architecture is identical between base model and distill, so the tensor override map transfers directly.
# Extract recipe from reference GGUF
from gguf import GGUFReader
r = GGUFReader('unsloth/Qwen3.6-27B-UD-Q5_K_XL.gguf')
for t in r.tensors:
if t.tensor_type.name not in ('Q5_K', 'F32'):
print(f'--tensor-type {t.name}={t.tensor_type.name}')
# → 195 overrides
llama-quantize \
--imatrix Qwen3.6-27B-Claude-Opus-Reasoning-Distilled.imatrix \
[195 --tensor-type overrides] \
rico03-distill-f16.gguf \
output-UD-Q5_K_XL.gguf \
Q5_K_M
Qwen3.6-27B hybrid SSM+Transformer:
Apache 2.0 — inherited from the base model.