QwenPaw-Flash-9B-heretic-MTP-GGUF

cvgro/QwenPaw-Flash-9B-heretic-MTP-GGUF

QwenPaw-Flash-9B-heretic-MTP-GGUF is a machine learning model from cvgro. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; margin-bottom: 30px;" <div style="background: f5f5f7; border-radius: 20px; padding: 36px 32px; margin-bottom: 20px; text-alig…

Downloads · 30 days

11% of all-time downloads

All-time downloads

606

Public

Repo size

42.5 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf42.5 GB · 100%

At a glance

License: apache-2.0
Access: Public
Created: Jun 29, 2026
Updated: Jun 29, 2026
SHA: 63e0abf7

Base models

Model card

From the Hugging Face model README

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; margin-bottom: 30px;"> <div style="background: #f5f5f7; border-radius: 20px; padding: 36px 32px; margin-bottom: 20px; text-align: center; position: relative; overflow: hidden;"> <div style="position: absolute; top: -30px; right: -30px; width: 120px; height: 120px; background: #ffedd5; border-radius: 50%;"></div> <div style="position: absolute; bottom: -20px; left: 40px; width: 80px; height: 80px; background: #fed7aa; border-radius: 50%;"></div> <div style="position: absolute; top: 50%; left: -15px; width: 60px; height: 60px; background: #ffedd5; border-radius: 50%;"></div> <div style="display: inline-flex; gap: 8px; margin-bottom: 16px; position: relative; z-index: 1;"> <span style="background: #007aff; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">MTP</span><span style="background: #af52de; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">GGUF</span> </div> <h1 style="margin: 0 0 8px 0; font-size: 32px; font-weight: 700; color: #1d1d1f; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">QwenPaw-Flash-9B-heretic-MTP</h1> <p style="margin: 8px 0 0 0; font-size: 14px; position: relative; z-index: 1;"><span style="color: #86868b;">English</span> | <a href="https://huggingface.co/SC117/QwenPaw-Flash-9B-heretic-MTP-GGUF/blob/main/README_zh.md" style="color: #007aff; text-decoration: none;">📖 中文文档</a></p> </div> </div> <div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; display: flex; flex-direction: column; gap: 20px; margin-bottom: 30px;"> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="padding: 16px;"> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">QwenPaw-Flash-9B-heretic non-MTP Version: <a href="https://huggingface.co/SC117/QwenPaw-Flash-9B-heretic-GGUF" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">QwenPaw-Flash-9B-heretic-GGUF</a></p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;"><p align="center"></p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;"><b>🏆 BenchLocal Total: 4035/5000 (80.7%) — MTP Speculative Decoding Injected</b><br></p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;"><i>Uncensored · Abliterated · Agent-Optimized · 1.7-4.1× Speedup</i></p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;"></p></p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">Uncensored version of <b>QwenPaw-Flash-9B</b>, processed with <a href="https://github.com/p-e-w/heretic" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">Heretic</a> v1.3.0 abliteration, with <b>MTP (Multi-Token Prediction) head</b> weights injected from the original Qwen3.5-9B base model.</p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">By reconstructing the MTP speculative decoding head — which was stripped during the QwenPaw fine-tuning process — this model achieves <b>up to 4.1× inference speedup</b> on real agent benchmarks while maintaining or improving accuracy.</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📊</span> 🏆 BenchLocal Benchmarks (With MTP)</div> <div style="padding: 16px;"> <p style="margin: 0 0 12px 0; font-size: 13px; color: #64748b; font-style: italic; border-left: 3px solid #ffedd5; padding-left: 12px;"><b>Test Environment</b>: NVIDIA RTX 5070 Ti (16GB) · llama.cpp (turboquant build, <code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">--spec-type draft-mtp</code>) · Q6_K quant</p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #64748b; font-style: italic; border-left: 3px solid #ffedd5; padding-left: 12px;"><b>Framework</b>: <a href="https://github.com/stevibe/benchlocal" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">BenchLocal</a> — local model agent evaluation suite</p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #64748b; font-style: italic; border-left: 3px solid #ffedd5; padding-left: 12px;"><b>Methodology</b>: Each scenario run <b>once</b>, no retries, no second attempts</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Benchmark</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Score</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Accuracy</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Results</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Time</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">vs No-MTP</th> </tr></thead><tbody> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>ToolCall-15</b> 🛠️</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1500/1500</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>100%</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">15✅ 0⚠️ 0❌</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">0.65min</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1.4×</b> faster</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>HermesAgent-20</b> 🤖</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1505/2000</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>75.3%</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">12✅ 1⚠️ 7❌</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">5.3min</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1.17×</b> faster</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>BugFind-15</b> 🐛</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1030/1500</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>68.7%</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">9✅ 2⚠️ 4❌</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">1.8min</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>4.1×</b> faster</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>Total</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>4035/5000</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>80.7%</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">36✅ 3⚠️ 11❌</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>7.8min</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1.9×</b> faster</td> </tr> </tbody></table> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">Comparison: With vs Without MTP</h3> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Benchmark</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Without MTP</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">With MTP</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Δ Score</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Δ Speed</th> </tr></thead><tbody> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>ToolCall-15</b> 🛠️</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">1400/1500 (93.3%)</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1500/1500 (100%)</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>+100 pts</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1.4×</b></td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>HermesAgent-20</b> 🤖</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">1545/2000 (77.2%)</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1505/2000 (75.3%)</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">−40 pts</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1.17×</b></td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>BugFind-15</b> 🐛</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">928/1500 (61.9%)</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1030/1500 (68.7%)</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>+102 pts</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>4.1×</b></td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>Total</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>3873/5000 (77.5%)</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>4035/5000 (80.7%)</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>+162 pts</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1.9×</b></td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>Total Time</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>14.7 min</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>7.8 min</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>1.9×</b></td> </tr> </tbody></table> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">🛠️ ToolCall-15 — Tool Calling Stability (100%, +6.7 pts)</h3> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">MTP speculative decoding eliminated the single failure (TC-05: Relative date/time parsing, which previously scored 0). All 15 scenarios now pass perfectly.</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">TC-ID</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Result</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Scenario</th> </tr></thead><tbody> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">TC-01–TC-04</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">Simple / Multi / Nested / Type conversion</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">TC-05</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">Relative date/time parsing ← <b>fixed by MTP</b></td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">TC-06–TC-15</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">All remaining scenarios</td> </tr> </tbody></table> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">🤖 HermesAgent-20 — Complex Agent Tasks (75.3%, −1.9 pts)</h3> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">MTP decoding introduces minor noise in long-chain reasoning scenarios (~40pt drop), likely because draft tokens occasionally derail the generation path in multi-step planning tasks. However, the speed gain (1.17×) and the fact that the drop is within noise range (single-run variance was 255pts for Qwopus MTP) makes this an acceptable trade-off.</p> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">🐛 BugFind-15 — Code Debugging (68.7%, +6.8 pts)</h3> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">Significant improvement — MTP's faster decoding effectively prevents timeout failures (BF-12 previously hit 300s timeout, now completes in time) and the draft context helps maintain debugging focus.</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">BF-ID</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Without MTP</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">With MTP</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Δ</th> </tr></thead><tbody> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-01</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-02</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 88</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">+12</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-03</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">❌ 0</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">❌ 0</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-04</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-05</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">❌ 40</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">⚠️ 70</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">+30</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-06</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">❌ 0</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">❌ 0</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-07</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-08</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-09</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-10</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">❌ 0</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">❌ 0</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-11</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">⚠️ 60</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">+40</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-12</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">❌ 0 (timeout)</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>+100</b></td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-13</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ 100</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-14</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">⚠️ 70</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">⚠️ 60</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">−10</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">BF-15</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">⚠️ 70</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">⚠️ 60</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">−10</td> </tr> </tbody></table> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>⚡</span> MTP Speculative Decoding</div> <div style="padding: 16px;"> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">What is MTP?</h3> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">Multi-Token Prediction (MTP) is a speculative decoding technique where a small "draft head" predicts multiple future tokens in parallel. The main model then verifies these drafts in a single forward pass, accepting correct predictions for up to 2-4× speedup in practice.</p> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">Injection Method</h3> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">The original Qwen3.5-9B base model ships with a 4-layer MTP head (~243M params) in its architecture configuration. During QwenPaw fine-tuning, the MTP head weights were stripped (only the config placeholder <code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">mtp_num_hidden_layers: 1</code> remained, but no actual tensors existed in the safetensors).</p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;"><b>Recovery process:</b></p> <ol style="margin: 0 0 12px 0; padding-left: 20px;"></ol> <ol style="margin: 0 0 12px 0; padding-left: 20px;"></ol> <ol style="margin: 0 0 12px 0; padding-left: 20px;"></ol> <ol style="margin: 0 0 12px 0; padding-left: 20px;"></ol> <ol style="margin: 0 0 12px 0; padding-left: 20px;"></ol> <ol style="margin: 0 0 12px 0; padding-left: 20px;"></ol> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;"><b>Total injected parameters:</b> 243.3M (2.7% of main model)</p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;"><b>MTP acceptance rate (draft-n-max=2):</b> ~50% (1083 accepted / 2166 generated across all benchmarks)</p> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">Why This Works</h3> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">The MTP head is a lightweight 4-layer MLP decoder that maps the main model's last hidden state to future token logits. It sits entirely in speculative decoding space — the main model's weights are unchanged, so no fine-tuning or retraining is needed. The head simply needs to exist with compatible dimensions for llama.cpp's <code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">--spec-type draft-mtp</code> to activate.</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>⚡</span> Comparison: QwenPaw MTP vs Other Models</div> <div style="padding: 16px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Model</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Total</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">ToolCall-15</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">HermesAgent-20</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">BugFind-15</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Total Time</th> </tr></thead><tbody> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>🐾 QwenPaw MTP 9B</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>4035 🥇</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>100%</b> 🥇</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">75.3%</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>68.7%</b></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>7.8min</b> 🥇</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">🐾 QwenPaw 9B (no MTP)</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">3873</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">93.3%</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><b>77.2%</b> 🥇</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">61.9%</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">14.7min</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">🧠 Qwopus 9B MTP</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">3935</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">93.3%</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">67.3% ⚠️</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">79.0% 🥇</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">21.3min ⚠️</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">🧠 Qwen 35B Thinking ON</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">1445 (HA only)</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">72.3%</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">7.0min</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">⚡ Qwen 35B Thinking OFF</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">1370 (HA only)</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">68.5%</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">5.1min</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">🔮 Gemma 4 26B</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">1405 (HA only)</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">70.3%</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">—</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">18.6min</td> </tr> </tbody></table> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;"><b>QwenPaw MTP wins</b> on 2/3 benchmarks + total score + total time. The only benchmark it loses is BugFind-15 (to Qwopus MTP), but Qwopus suffers from severe instability (255pt variance on HermesAgent-20, with a worst-case 6.2min timeout).</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🧠</span> Model Description</div> <div style="padding: 16px;"> <ul style="margin: 0 0 12px 0; padding-left: 20px;"><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Base model**: <a href="https://huggingface.co/agentscope-ai/QwenPaw-Flash-9B" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">QwenPaw-Flash-9B</a> (Qwen3.5-9B fine-tuned for autonomous agent scenarios)</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">MTP head source**: <a href="https://huggingface.co/Qwen/Qwen3.5-9B" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">Qwen/Qwen3.5-9B</a> (original base model, layer 32 MTP head)</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Tool**: Heretic v1.3.0 (automatic directional ablation)</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Best trial**: #194 / 230 trials (abliteration)</li></ul> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>⚙️</span> Abliteration Parameters</div> <div style="padding: 16px;"> <p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">direction_index = 21.13 attn.o_proj.max_weight = 1.42 attn.o_proj.max_weight_position = 21.72 attn.o_proj.min_weight = 1.11 attn.o_proj.min_weight_distance = 18.14 mlp.down_proj.max_weight = 1.48 mlp.down_proj.max_weight_position = 21.23 mlp.down_proj.min_weight = 1.47 mlp.down_proj.min_weight_distance = 17.47</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🏗️</span> Architecture</div> <div style="padding: 16px;"> <ul style="margin: 0 0 12px 0; padding-left: 20px;"><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Type**: Qwen3_5ForConditionalGeneration (multimodal with vision encoder) + MTP spec head</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Main model parameters**: ~9B</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">MTP head parameters**: ~243M (2.7% overhead)</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Layers**: 32 (hybrid: Gated DeltaNet + Gated Attention) + 4 MTP decoder layers</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Context length**: 262,144 tokens</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Speculative decoding**: <code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">--spec-type draft-mtp</code> with <code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">--spec-draft-n-max 2</code></li></ul> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📦</span> GGUF Files</div> <div style="padding: 16px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">File</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Size</th> <th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Notes</th> </tr></thead><tbody> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">QwenPaw-Flash-9B-heretic-MTP-Q8_0.gguf</code></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">~9.2GB</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">High quality, near lossless</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">QwenPaw-Flash-9B-heretic-MTP-Q6_K.gguf</code></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">~7.1GB</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">✅ <b>Recommended, best value</b></td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">QwenPaw-Flash-9B-heretic-MTP-Q4_K_M.gguf</code></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">~5.4GB</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">Compact</td> </tr> <tr> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;"><code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">mmproj-BF16</code></td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">~880MB</td> <td style="padding: 7px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); color: #334155; text-align: left;">Vision encoder (multimodal) — same as non-MTP version</td> </tr> </tbody></table> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> Usage</div> <div style="padding: 16px;"> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">--spec-type draft-mtp</p> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">--spec-draft-n-max 2</p> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">llama.cpp (with MTP speculative decoding)</h3> <p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;"># Start server with MTP enabled llama-server -m QwenPaw-Flash-9B-heretic-MTP-Q6_K.gguf \ -ngl 99 -fa on -c 8192 \ --spec-type draft-mtp --spec-draft-n-max 2 \ --host 0.0.0.0 --port 8088

Or with CLI

llama-cli -m QwenPaw-Flash-9B-heretic-MTP-Q6_K.gguf
-ngl 99 -fa on -c 8192
--spec-type draft-mtp --spec-draft-n-max 2
-p "Write a Python script to..."</p>

<h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">llama.cpp (without MTP, fallback)</h3> <p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;"># The model works as a normal GGUF too — just omit spec args llama-server -m QwenPaw-Flash-9B-heretic-MTP-Q6_K.gguf \ -ngl 99 -fa on -c 8192 \ --host 0.0.0.0 --port 8088</p> <h3 style="margin: 16px 0 8px 0; font-size: 14px; color: #1e293b; font-weight: 700;">LM Studio</h3> <p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">Load the GGUF file directly. For MTP speculative decoding, LM Studio must support <code style="background: #f8fafc; padding: 2px 6px; border-radius: 4px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b;">--spec-type</code> — if not, the model functions as a standard 9B model.</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📝</span> Notes</div> <div style="padding: 16px;"> <ol style="margin: 0 0 12px 0; padding-left: 20px;"><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Safety filters have been significantly reduced via abliteration</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">KL divergence is only 0.0225 — minimal impact on model intelligence</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">The original model supports multimodal (vision); GGUF versions require the mmproj file from the non-MTP release</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">BenchLocal scores measured at <b>Q6_K</b> on RTX 5070 Ti 16GB with llama.cpp (turboquant). Each scenario was run <b>once with no retries</b></li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">MTP acceptance rate of ~50% at draft-n-max=2 means ~25-40% wall-clock speedup on short prompts, and up to 4× on long-generation tasks (debugging, code writing)</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">BugFind-15 saw the largest improvement (4.1×) because debugging tasks are generation-heavy — more tokens, more drafts accepted</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">The MTP head is a <b>lossless copy</b> from the original Qwen3.5-9B — no training was involved, simply weight injection</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Agent-heavy scenarios (HermesAgent-20) see the least MTP benefit because short-turn interactions don't give the draft head enough runway</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;">Please use responsibly</li></ol> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🙏</span> Acknowledgements</div> <div style="padding: 16px;"> <ul style="margin: 0 0 12px 0; padding-left: 20px;"><li style="margin-bottom: 4px; font-size: 13px; color: #334155;"><a href="https://github.com/p-e-w/heretic" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">Heretic</a> — Automated censorship removal</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;"><a href="https://huggingface.co/agentscope-ai/QwenPaw-Flash-9B" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">agentscope-ai/QwenPaw-Flash-9B</a> — Base model</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;"><a href="https://huggingface.co/Qwen/Qwen3.5-9B" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">Qwen/Qwen3.5-9B</a> — MTP head source</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;"><a href="https://github.com/ggml-org/llama.cpp" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">llama.cpp</a> — GGUF quantization and inference</li><li style="margin-bottom: 4px; font-size: 13px; color: #334155;"><a href="https://github.com/stevibe/benchlocal" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">BenchLocal</a> — Local model agent evaluation suite</li></ul> </div> </div> </div>

License: apache-2.0
Languages: en, zh
Created: Jun 29, 2026
Updated: Jun 29, 2026