Qwen3.6-27B-abliterated-AutoRound-INT4-MTP

<div style="background:#0b1020; border:1px solid #1e293b; border-top:4px solid #a855f7; border-radius:14px; padding:26px 28px; margin-bottom:22px; font-family:ui-monospace,SFMono-Regular,Menlo,monospace;"> <div style="color:#e2e8f0; font-size:12px; letter-spacing:2px;">// ANTONYMOX · UNCENSORED BUILD</div> <div style="color:#ffffff; font-size:27px; font-weight:800; line-height:1.2; margin:8px 0 6px; font-family:-apple-system,Segoe UI,Roboto,sans-serif;">Qwen3.6-27B <span style="color:#c084fc;">abliterated</span> — INT4 · MTP</div> <div style="color:#94a3b8; font-size:13.5px; line-height:1.6; max-width:640px; font-family:-apple-system,Segoe UI,Roboto,sans-serif;">Decensored without losing brains. Keeps its MTP head (2× decode), runs up to 256k context on one card, reads images, and ships the benchmark JSON — not just a claim.</div> <div style="margin-top:15px; display:flex; flex-wrap:wrap; gap:7px;"> <span style="color:#c4b5fd; border:1px solid #334155; border-radius:6px; padding:3px 9px; font-size:11px;">KL&nbsp;0.0089</span> <span style="color:#86efac; border:1px solid #334155; border-radius:6px; padding:3px 9px; font-size:11px;">MTP&nbsp;preserved</span> <span style="color:#7dd3fc; border:1px solid #334155; border-radius:6px; padding:3px 9px; font-size:11px;">256k&nbsp;ctx</span> <span style="color:#f9a8d4; border:1px solid #334155; border-radius:6px; padding:3px 9px; font-size:11px;">vision</span> <span style="color:#fdba74; border:1px solid #334155; border-radius:6px; padding:3px 9px; font-size:11px;">19&nbsp;GB</span> <span style="color:#e2e8f0; border:1px solid #334155; border-radius:6px; padding:3px 9px; font-size:11px;">Apache-2.0</span> </div> </div> <div style="display:flex; flex-wrap:wrap; background:#ffffff; border:1px solid #e2e8f0; border-radius:12px; overflow:hidden; margin-bottom:24px; font-family:-apple-system,Segoe UI,Roboto,sans-serif;"> <div style="flex:1; min-width:110px; padding:16px 10px; text-align:center; border-right:1px solid #eef2f7;"> <div style="font-size:26px; font-weight:800; color:#7c3aed;">70</div> <div style="font-size:10.5px; color:#64748b; letter-spacing:0.8px;">LCB-HARD</div> </div> <div style="flex:1; min-width:110px; padding:16px 10px; text-align:center; border-right:1px solid #eef2f7;"> <div style="font-size:26px; font-weight:800; color:#059669;">87</div> <div style="font-size:10.5px; color:#64748b; letter-spacing:0.8px;">TOOL-USE</div> </div> <div style="flex:1; min-width:110px; padding:16px 10px; text-align:center; border-right:1px solid #eef2f7;"> <div style="font-size:26px; font-weight:800; color:#0891b2;">0.0089</div> <div style="font-size:10.5px; color:#64748b; letter-spacing:0.8px;">KL DRIFT</div> </div> <div style="flex:1; min-width:110px; padding:16px 10px; text-align:center; border-right:1px solid #eef2f7;"> <div style="font-size:26px; font-weight:800; color:#db2777;">115</div> <div style="font-size:10.5px; color:#64748b; letter-spacing:0.8px;">TOK/S · MTP4</div> </div> <div style="flex:1; min-width:110px; padding:16px 10px; text-align:center;"> <div style="font-size:26px; font-weight:800; color:#334155;">19<span style="font-size:14px;">GB</span></div> <div style="font-size:10.5px; color:#64748b; letter-spacing:0.8px;">ON DISK</div> </div> </div> <div style="text-align:center; font-size:11px; color:#94a3b8; margin:-16px 0 24px; font-family:-apple-system,Segoe UI,Roboto,sans-serif;">📊 All benchmarks &amp; throughput measured on a single <b>RTX 4090 (48 GB)</b> · vLLM 0.19 · MTP n=4 · seed 42</div>

A gentle Heretic abliteration of the pristine Qwen/Qwen3.6-27B → AutoRound INT4 → the 15-tensor MTP head grafted back in BF16 so speculative decoding works out of the box in vLLM. Removing refusals cost zero hard-coding points (70 = 70 vs the clean base) and tool-use went up (87 vs 83). Pick a preset, run one script, done.

---

Highlights

<div style="display:grid; grid-template-columns:repeat(auto-fit,minmax(160px,1fr)); gap:10px; margin-bottom:26px; font-family:-apple-system,Segoe UI,Roboto,sans-serif;"> <div style="border:1px solid #e9d5ff; border-radius:10px; padding:14px; background:#ffffff;"> <div style="font-size:14px; font-weight:800; color:#7c3aed;">🔓 Uncensored, sharp</div> <div style="font-size:11.5px; color:#64748b; margin-top:4px; line-height:1.5;">refusals gone, hard score <b>unchanged</b> (70 = 70)</div> </div> <div style="border:1px solid #bbf7d0; border-radius:10px; padding:14px; background:#ffffff;"> <div style="font-size:14px; font-weight:800; color:#059669;">🛠️ Better tool-use</div> <div style="font-size:11.5px; color:#64748b; margin-top:4px; line-height:1.5;">87 vs 83 — beats the clean base</div> </div> <div style="border:1px solid #a5f3fc; border-radius:10px; padding:14px; background:#ffffff;"> <div style="font-size:14px; font-weight:800; color:#0891b2;">⚡ ~2× decode</div> <div style="font-size:11.5px; color:#64748b; margin-top:4px; line-height:1.5;">MTP head kept → spec-decode works</div> </div> <div style="border:1px solid #bae6fd; border-radius:10px; padding:14px; background:#ffffff;"> <div style="font-size:14px; font-weight:800; color:#0369a1;">📏 256k context (native)</div> <div style="font-size:11.5px; color:#64748b; margin-top:4px; line-height:1.5;">supports 256k; up to 256k fits on a 48 GB card</div> </div> <div style="border:1px solid #fbcfe8; border-radius:10px; padding:14px; background:#ffffff;"> <div style="font-size:14px; font-weight:800; color:#be185d;">🖼️ Multimodal</div> <div style="font-size:11.5px; color:#64748b; margin-top:4px; line-height:1.5;">images &amp; video · vision tower full BF16</div> </div> <div style="border:1px solid #ddd6fe; border-radius:10px; padding:14px; background:#ffffff;"> <div style="font-size:14px; font-weight:800; color:#6d28d9;">🧠 Reads it all</div> <div style="font-size:11.5px; color:#64748b; margin-top:4px; line-height:1.5;">7/8 exact recall @ 150k tokens</div> </div> <div style="border:1px solid #fde68a; border-radius:10px; padding:14px; background:#ffffff;"> <div style="font-size:14px; font-weight:800; color:#b45309;">🎯 One-click presets</div> <div style="font-size:11.5px; color:#64748b; margin-top:4px; line-height:1.5;">24 / 32 / 48 GB — measured, not guessed</div> </div> <div style="border:1px solid #cbd5e1; border-radius:10px; padding:14px; background:#ffffff;"> <div style="font-size:14px; font-weight:800; color:#0f172a;">🧾 Proof in-repo</div> <div style="font-size:11.5px; color:#64748b; margin-top:4px; line-height:1.5;">the per-task benchmark JSON ships with it</div> </div> </div>

---

Benchmarks — decensoring cost nothing, and where it sits

Same base Qwen/Qwen3.6-27B, one harness. Decensoring left hard unchanged (70 = 70 vs the reference narrow quant) and tool-use went up (83 → 87):

[!NOTE] Numbers are % of tasks passed (40 hard coding + 30 tool tasks; raw pass/total in benchmark-results/). Conditions: temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 · seed 42 · non-thinking · vLLM 0.19 · RTX 4090 (48 GB) · MTP n=4. Solutions run in a restricted sandbox (no network) — safe, identical scoring. The Lorbus row is our own measurement of their public quant on the same tasks, not their claim. The 80B Coder-Next row is likewise our own measurement of the UD-IQ4_XS GGUF (via LM Studio) on the same task set — a bigger, different model shown for scale. Reproduce it, don't trust it.

---

Surgically gentle abliteration

Abliteration strips the refusal direction — done carelessly it also scrambles the model's brains. Heretic co-minimizes refusals and KL divergence from the original. Lower KL = closer to the untouched model = intelligence kept. We tuned for minimum drift:

~18× below Heretic's own showcase, 50–100× below manual. The model barely moved from the base, so <think> reasoning and coding stayed intact. (KL is model/harness-dependent, so cross-model figures aren't a strict race — the order-of-magnitude gap is the point.) We kept 11/100 refusals as the price of rock-bottom drift — brains first.

Kept in BF16 (not quantized) — surgical precision

Everything heavy is 4-bit; only the fragile bits that carry reasoning, speed and vision stay full-precision.

---

Speed — MTP speculative decoding

Honest 768-token × 3-run measurement on a single RTX 4090 (48 GB), vLLM 0.19:

Use num_speculative_tokens: 4. Speculative decoding is lossless — n changes speed only, never output. All presets use n=4.

---

Multimodal — sees images & video

The full vision-language model, not a text-only trim. The entire vision tower (333 tensors) is kept in full BF16 — quantization never touched it — so image/video understanding is identical to the pristine base. Serve with --limit-mm-per-prompt '{"image":4,"video":1}'. Published benchmarks cover text/coding/tool-use; vision quality is inherited, not separately re-scored.

---

Quickstart — pick your GPU

vllm serve antonyMox/Qwen3.6-27B-abliterated-AutoRound-INT4-MTP \
  --quantization auto-round \
  --max-model-len 40960 \
  --gpu-memory-utilization 0.95 \
  --max-num-seqs 1 \
  --kv-cache-dtype fp8_e4m3 \
  --enable-prefix-caching \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":4}' \
  --trust-remote-code

Long context is cheap here: Qwen3.6 is a hybrid (Mamba/GDN + attention), so KV costs only ~37 MB/1k tokens at fp8. The model's native maximum is 256k (262,144) tokens — how much of it fits depends on how much VRAM you give vLLM:

Context vs mem-util (48 GB card, measured)

Our default preset is a deliberate 0.60 / 200k — it keeps ~19 GB free so you can load a second model on the same card. Want the full 256k? Just raise --gpu-memory-utilization to ~0.65–0.68.

---

⚠️ Pitfall — never set --max-model-len at exact KV capacity

[!WARNING] On vLLM 0.19 hybrid models: if max-model-len equals KV capacity (startup log shows Maximum concurrency ... : 1.00x), requests in the last ~3 % of the window pass validation then hang forever — no response, no error, nothing logged, GPU idle. The scheduler can't allocate the final KV block (draft tokens need headroom the validator ignores). Above the window → instant HTTP 400 (fine); below the danger zone → fine. Only the top sliver dead-locks.

[!TIP] Keep --max-model-len ≥ 2 attention blocks (2 × 1632) below capacity — our presets already do. Check the log: Maximum concurrency for N tokens: 1.13x must be ≥ ~1.10x, not 1.00x. Also: the log line GPU KV cache size: X tokens understates real capacity ~3× on hybrids (it divides by KV-cache groups) — trust only the Maximum concurrency line.

---

Long-context recall (measured)

Needle test on a 150k-token prompt (48 GB preset), exact-quote recall by depth:

7/8 exact. The miss (*) at 50% returned the neighbouring line — mild "lost-in-the-middle", not a hallucination. Prefix caching works: a repeat 150k query answered in 5.5 s vs 131 s cold.

---

🧬 The family

Clean quants of the same base — pick by hardware:

• wide · 26 GB · hard 78 🏆 — maximum reasoning, for 48 GB+ cards.
• abliterated · 19 GB · hard 70 (this one) — uncensored, fits 24 GB cards.
• 35B-A3B · 22 GB · hard 48 (coming soon) — fast MoE (~188 tok/s).

All ours: AutoRound INT4, MTP preserved, multimodal, published benchmarks. Measured against the community reference Lorbus narrow INT4 (hard 70) — see the comparison table above.

---

Recipe

1. Base — pristine Qwen/Qwen3.6-27B BF16.
2. Abliteration — Heretic, gentle: KL 0.0089, refusals 11/100, <think> intact.
3. Quant — AutoRound W4A16, group 128, 200 iters, dense. BF16-kept: mtp.*, visual.*, Mamba control.
4. MTP — 15 mtp.* tensors grafted from the pristine BF16.
5. Sampler — ships temp 1.0 / top_p 0.95 / top_k 20 (generation_config.json).

---

Why HuggingFace shows "~6.7B params"

Not a small model — it's how the widget reads a packed INT4 quant. The 800 quantized tensors pack 8× int4 into each int32, so HF counts 3.06 B int32 slots, not the ~24 B real int4 weights. With the BF16 parts, the true model is Qwen3.6-27B, 19 GB. Every AutoRound/GPTQ INT4 looks like this. Verified: language_model 4.55 B · other 1.27 B · vision 0.46 B · mtp 0.42 B.

---

⚠️ Usage & responsibility

[!CAUTION] This is an uncensored research artifact — read before use.

• Reduced refusals by design — can produce content a safety-aligned model would decline.
• No safety guarantees — outputs are not filtered; review before use, especially in production.
• You are responsible — for lawful, ethical use in your jurisdiction; all consequences rest with you.
• No liability — provided "as is" as a community artifact, no warranty of any kind.
• Not for harm — for research, evaluation, and lawful applications only.

License

Apache-2.0, same as the base. Quantized & abliterated from Qwen/Qwen3.6-27B. Community build; benchmark numbers from our own harness — JSON included, methodology open.