Downloads · 30 days
895
2% of all-time downloads
Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF
Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF is a text generation model from Brian6145. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
A distilled Qwen3.6-27B GGUF optimized for local agentic reasoning, tool use, and long-chain task execution.
Downloads · 30 days
895
2% of all-time downloads
All-time downloads
48.8K
Public
Repo size
147 GB
Likes
22
Public
Click a slice to open those files.
.gguf147 GB · 100%
From the Hugging Face model README
A distilled Qwen3.6-27B GGUF optimized for local agentic reasoning, tool use, and long-chain task execution.
Please ensure
temperature = 0.6andtop_p = 0.95when using this model.This model was trained and validated at these specific parameters. Both too high and too low temperatures cause problems:
🔥 Too high (> 0.6) → Output becomes divergent as token probabilities flatten. This causes malformed tool calls, function name hallucinations, unstable parameter generation, and uncontrollable agent behavior.
🧊 Too low (< 0.3) → The model almost always picks the highest-probability token. In tool-calling scenarios this manifests as: repeating the same failed tool call instead of trying alternatives, shortened thinking traces leading to insufficient reasoning depth, and reduced robustness to edge cases.
This is an important recommendation based on extensive real-world testing. Please verify these parameters in your inference framework.
| Area | Score | vs Qwen3.6-27B q4_k_m |
|---|---|---|
| BenchLocal 6-pack 🏆 | 86.5 | +8.3 |
| GPQA-Diamond-198 🔬 | 83.84% | +10.14% |
| BugFind-15 🐛 | 80 | +20 |
| ToolCall-15 🔧 | 97 | +4 |
| InstructFollow-15 📋 | 94 | +17 |
| StructOutput-15 📊 | 88 | +11 |
| MMLU-500 (5-shot) 📚 | 91.80% | ~tied (+0.2%) |
| DataExtract-15 📄 | 81 | -2 |
🚀 Output speed:
~60 tok/son A100 40GB ·~100 tok/son RTX PRO 6000 (q4_k_m + mtp=3)
The original Qwen3.6-27B has solid foundational capabilities, but its agent behavior falls short — prone to infinite loops when thinking, lacks structured agent design, and has room to improve in math reasoning.
This variant tackles all three through targeted distillation:
The result is a local agent that doesn't just score high on benchmarks — it works reliably in real engineering tasks like BugFind, where it now competes with GLM5.2.
⚠️ Note: Side-by-side GLM5.2 comparisons were evaluated using Opus 4.8 as a judge. Opus-as-judge has inherent biases — results are indicative, not definitive.
Core insight: Qwen3.6-27B doesn't lack capability — it lacks good agent behavior. That makes it worth iterating on.
We followed the ReAct paradigm (Yao et al., ICLR 2023) — unifying reasoning and action into an alternating, constrained, executable loop — rather than just making the model "think longer" or "call tools better."
| Teacher Model | Capability Distilled |
|---|---|
| Claude Opus 🎯 | Systematic thinking, structured organization, concise reasoning |
| DeepSeek 🧭 | Stable agent behavior, tool orchestration, task closure |
| Math/Logic models ➗ | Mathematical reasoning, logical deduction |
| Pack | q4_k_m (ours) | Qwen/Qwen3.6-27B q4_k_m | Delta |
|---|---|---|---|
| BugFind-15 🐛 | 80 | 60 | +20 |
| ToolCall-15 🔧 | 97 | 93 | +4 |
| DataExtract-15 📄 | 81 | 83 | -2 |
| InstructFollow-15 📋 | 94 | 77 | +17 |
| ReasonMath-15 ➗ | 79 | 79 | 0 |
| StructOutput-15 📊 | 88 | 77 | +11 |
| Total 🏆 | 86.5 | 78.2 | +8.3 |
| Benchmark | Ours | Baseline | Notes |
|---|---|---|---|
| GPQA-Diamond-198 🔬 | 83.84% | 73.7% | +10.14%, all 198 graded locally |
| MMLU-500 (5-shot) 📚 | 91.80% | 91.6% | Approximately tied |
🧩 OpenCode + LM Studio
📐 Temperature: 0.6 · Top-p: 0.95
⚡ q4_k_m + mtp=3
# Download the GGUF
huggingface-cli download your-org/Opus-DeepSeek-Distilled-Q4M \
opus-deepseek-distilled-q4m-q4_k_m.gguf --local-dir ./models
# Run with llama.cpp
./llama-cli -m ./models/opus-deepseek-distilled-q4m-q4_k_m.gguf \
--temp 0.6 --top-p 0.95 \
-p "Your prompt here"
temperature = 0.6, top_p = 0.95💡 Pro tip: For coding tasks, the
temp 0.6 / top_p 0.95combo delivers the best balance of creativity and correctness.
q4_k_m tested; q5_k_m may improve accuracy at higher VRAM cost@misc{opus-deepseek-distilled-q4m,
title = {Opus-DeepSeek-Distilled-Q4M: A Distilled Agentic GGUF for Local Deployment},
author = {Yin, Brian and BenchLocal Contributors},
year = {2026},
url = {https://github.com/brianyin/BenchLocal}
}
It is because of this work that came before that we can continue pushing forward, arriving at today's more stable, more practical, and more complete agent — and helping us get closer to the era of local agent AI.