Downloads · 30 days
377
2% of all-time downloads
Brian6145/Qwen3.6-27B-Claude-Opus-Sonnet-DistilledV2-MTP-GGUF
Qwen3.6-27B-Claude-Opus-Sonnet-DistilledV2-MTP-GGUF is a text generation model from Brian6145. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
GGUF quantized release of the Claude Opus / Sonnet reasoning distillation on Qwen3.6-27B, with native MTP speculative decoding support in llama.cpp.
Downloads · 30 days
377
2% of all-time downloads
All-time downloads
23.3K
Public
Repo size
79.2 GB
Likes
15
Public
Click a slice to open those files.
.gguf68.3 GB · 100%
From the Hugging Face model README
GGUF quantized release of the Claude Opus / Sonnet reasoning distillation on Qwen3.6-27B, with native MTP speculative decoding support in llama.cpp.
Key numbers: Q4_K_M + MTP2 → 114.78 tok/s generation, 80.33% draft acceptance, 64% faster than non-MTP baseline. On the same machine, this release delivers 2x the visible answer content vs the original qwen3.6-27b while maintaining 4/4 correctness.
| File | Size | Best for |
|---|---|---|
| Q4_K_M (recommended) | 15.66 GB | Best overall balance |
| Q6_K | 20.89 GB | Quality-first |
| Q2_K | 10.12 GB | Extreme compression |
| Q8_0 | 27.05 GB | High-fidelity experiments |
Same-machine benchmark against the original (non-quantized) qwen3.6-27b:

GGUF side includes llama-cli cold start — this is a conservative estimate.
| Original | This release | |
|---|---|---|
| Average response time | 10.93s | 10.09s |
| Correctness (4 prompts) | 3/4 | 4/4 |
| Visible answer chars | 1336 | 2845 |
| Hidden reasoning overhead | 9002 chars | minimal |
The original spends a large fraction of its token budget on hidden reasoning chains. This release converts that budget into visible answers, making it better suited for interactive local use.
Requires a recent llama.cpp build with Qwen3.5/3.6 MTP support. Older conversion pipelines may miss the required metadata and fail with failed to create MTP context.
Verified stack:
llama.cpp-ngl 999 --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-ngl 999Test: three-person logic puzzle, n=160, GPU + MTP2.
| Variant | Prompt tok/s | Generation tok/s | Draft acceptance |
|---|---|---|---|
| Q2_K + MTP2 | 439.73 | 118.01 | 68.66% |
| Q4_K_M + MTP2 | 240.55 | 114.78 | 80.33% |
| Q6_K + MTP2 | 503.87 | 99.85 | 78.86% |
| Q8_0 + MTP2 | 421.04 | 78.86 | 69.17% |
MTP vs non-MTP baseline (Q4_K_M):
| Variant | Prompt tok/s | Generation tok/s |
|---|---|---|
| Non-MTP | 796.22 | 69.98 |
| MTP2 | 240.55 | 114.78 |
| MTP3 | 390.77 | 117.16 |
MTP2 offers the best acceptance/throughput tradeoff. MTP3 acceptance drops to 69.48%.
Prompt lengths ~6.6K (ctx8k) and ~26.7K (ctx32k). Generation is intentionally short (17-23 tokens) to isolate prompt processing.
| Context | Variant | Prompt tok/s | Generation tok/s | Draft acceptance |
|---|---|---|---|---|
| ctx8k | Q2_K | 1304.11 | 104.41 | 83.33% |
| ctx8k | Q4_K_M | 2798.63 | 31.73 | 60.00% |
| ctx8k | Q6_K | 2415.74 | 69.48 | 60.00% |
| ctx8k | Q8_0 | 2143.06 | 63.78 | 60.00% |
| ctx32k | Q2_K | 2450.46 | 71.41 | 78.57% |
| ctx32k | Q4_K_M | 2846.65 | 87.42 | 83.33% |
| ctx32k | Q6_K | 2620.59 | 81.02 | 71.43% |
| ctx32k | Q8_0 | 3120.27 | 71.19 | 71.43% |
Q4_K_M is the most balanced variant across both short and long contexts. Q6_K is a solid quality-first choice.
Note: BF16 + MTP2 (historical reference) yielded 20.49 tok/s prompt / 0.85 tok/s generation on this GPU — quantization is required for practical throughput on this hardware.
Upgrade to LM Studio 0.4.14 or later. Load the GGUF file and MTP speculative decoding is enabled automatically — no settings, no flags, no configuration needed.
# Regular inference
./llama-cli -m Qwen3.6-27B-Claude-Opus-Sonnet-DistilledV2-MTP-Q4_K_M.gguf -ngl 999 -c 8192 -p "Your prompt here"
# With MTP enabled
./llama-cli -m Qwen3.6-27B-Claude-Opus-Sonnet-DistilledV2-MTP-Q4_K_M.gguf -ngl 999 -c 8192 \
--spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-ngl 999 -p "Your prompt here"
Recommended args:
-c 4096 --temp 0 --top-k 1 --spec-draft-n-max 2-c 8192 --temp 0 --top-k 1 --spec-draft-n-max 2All four quantized variants passed:
draft-mtp loadability| Variant | Quality verdict | Recommendation |
|---|---|---|
| Q2_K | Usable, most aggressive compression | Extreme compression only |
| Q4_K_M | Best balance | Default |
| Q6_K | More stable quality | Quality-first choice |
| Q8_0 | Fine, but not always faster than Q6_K | High-fidelity experiments |
Note: Windows PowerShell CLI may corrupt Chinese prompt arguments. Use UTF-8 prompt files, API calls, or your own inference service for Chinese workloads.
llama.cpp build (older exports may miss Qwen3.5/3.6 MTP metadata)V2 optimizes distillation targets, reasoning chain compression, and MTP deployment compatibility. Coding accuracy, tool calling stability, and debugging efficiency are all meaningfully improved.