Downloads · 30 days
68
8% of all-time downloads
Qwopus3.6-27B-Coder-abliterated-NVFP4
maci0/Qwopus3.6-27B-Coder-abliterated-NVFP4
Qwopus3.6-27B-Coder-abliterated-NVFP4 is a image-text-to-text model from maci0. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
<div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;max-width:760px;border:1px solid cec9ba;background:f2efe6;color:14130f;padding:24px 26px;border-radius:2px;margin-bottom:26px;"<div style="displ…
All-time downloads
865
Public
Parameters
26.9B
20.6 GB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors20.6 GB · 100%
Parameter types
How the weights are stored.
U824.4B · 91%
Try a prompt
Base models
Spaces
Model card
From the Hugging Face model README
<div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;max-width:760px;border:1px solid #cec9ba;background:#f2efe6;color:#14130f;padding:24px 26px;border-radius:2px;margin-bottom:26px;"><div style="display:flex;justify-content:space-between;align-items:center;gap:10px 14px;flex-wrap:wrap;"><span style="display:inline-flex;align-items:center;gap:9px;"><svg width="18" height="18" viewBox="0 0 24 24" aria-hidden="true" style="flex-shrink:0;"><path d="M4 4 H16 L20 8 V20 H4 Z" fill="none" stroke="#6f6b60" stroke-width="1.25"/><path d="M16 4 V8 H20" fill="none" stroke="#6f6b60" stroke-width="1.25"/><circle cx="9" cy="13" r="1" fill="#6f6b60"/><circle cx="13" cy="15" r="1" fill="#6f6b60"/></svg><span style="font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:12px;font-weight:700;letter-spacing:0.02em;">RQ-27B-CODER-A</span></span><span style="font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:10px;font-weight:700;letter-spacing:0.12em;text-transform:uppercase;color:#b5231c;border:1.5px solid #b5231c;border-radius:2px;padding:2px 7px;display:inline-block;transform:rotate(-3deg);">Abliterated</span></div><div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:26px;font-weight:800;letter-spacing:-0.02em;line-height:1.15;margin:18px 0 8px;color:#14130f;">Qwopus3.6-27B-Coder abliterated · NVFP4</div><div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:14.5px;line-height:1.5;color:#4a4740;margin-bottom:18px;">27B 256K agentic coder · tool-calling · refusals removed (2-round Heretic).</div><table style="width:100%;border-collapse:collapse;border:0;margin:2px 0 0;font-variant-numeric:tabular-nums;"><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-top:1.5px solid #14130f;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Params</th><td style="padding:9px 0;border:0;border-top:1.5px solid #14130f;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">27B</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Active</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">27B (dense)</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Size</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">18 GB</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Perplexity</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">6.73</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Refusals</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#b5231c;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">99 → 3 / 100</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Context</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">256K</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">MTP head</th><td style="padding:9px 0;border:0;background:none;text-align:right;font-weight:700;color:#b5231c;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">bf16</td></tr></table></div>
TL;DR: Qwopus3.6-27B-Coder abliterated, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 18 GB, wikitext-2 PPL 6.73, 256K agentic coder, refusals removed (2-round Heretic).
Qwopus3.6-27B-Coder abliterated NVFP4
Jackrong/Qwopus3.6-27B-Coder,
abliterated (refusal direction removed) with Heretic in
two iterative rounds, then quantized to NVFP4 (W4A4) in the compressed-tensors
nvfp4-pack-quantized format with llm-compressor
(GPTQ + MSE, shared fused-layer scales).
Near-lossless and decensored. Two Heretic rounds cut refusals from 99/100 to 3/100 of held-out harmful prompts while keeping a KL divergence of 0.0398 to the original model (well under the 0.5 line that signals capability damage). NVFP4 then compresses to ~18 GB with a wikitext-2 perplexity of 6.73.
- Built for vLLM on NVIDIA Blackwell (4-bit weight + 4-bit activation). Pre-Blackwell GPUs run it weight-only.
- Loading and generation verified in vLLM on an NVIDIA GB10 (Blackwell, sm_121).
Uncensored / abliterated model. It follows instructions without refusal guardrails. The abliteration only removes refusals; all other behaviour comes from the base model. You are responsible for how you use it.
Fidelity
Near-lossless versus the bf16 source, 18 GB vs 55.6 GB bf16 (~33%), at wikitext-2 perplexity 6.73 and KL divergence 0.0398 to the original. GPTQ error compensation and an MSE observer keep the drop from bf16 minimal; the header lists the full characteristics and Quantization covers the recipe.
Quickstart
NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag
needed. --reasoning-parser qwen3 splits the <think> block into reasoning_content;
--tool-call-parser qwen3_coder enables tool/function calling for agentic coding.
vllm serve maci0/Qwopus3.6-27B-Coder-abliterated-NVFP4 \
--served-model-name qwopus-27b-coder-abliterated-nvfp4 \
--max-model-len 131072 \
--gpu-memory-utilization 0.90 \
--kv-cache-dtype fp8 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder
- Supports up to 262144 tokens; keep at least 128K to preserve thinking quality.
--max-model-len 131072is a safe default; raise it if memory allows. - Add
--language-model-onlyto skip the vision tower and free KV cache for text use. - The parser flags are not auto-detected; pass them explicitly.
Python (OpenAI client)
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
r = client.chat.completions.create(
model="qwopus-27b-coder-abliterated-nvfp4",
messages=[{"role": "user", "content": "Write a Python function that merges two sorted lists."}],
)
print(r.choices[0].message.content)
curl
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "qwopus-27b-coder-abliterated-nvfp4",
"messages": [{"role": "user", "content": "Write a Python function that merges two sorted lists."}]
}'
About the base model
A 27B Qwen3.5-family vision-language model specialized for code (Qwopus 3.6 Coder), with thinking-mode reasoning and a 256K context window.
- 64 decoder layers: hybrid gated delta-net linear attention plus full attention, dense MLP, plus a vision tower for image and video input.
- 256K context (
max_position_embeddings262144). - Thinking mode by default, with an instruct toggle.
Abliteration
Heretic removes the refusal direction with a TPE-optimized
search over per-component ablation strength, jointly minimizing refusal rate and KL divergence
from the original model, then merges the best trial. This model was abliterated in two iterative
rounds: round 1 removed the dominant refusal direction, then Heretic was re-run on the round-1
model to remove the residual refusal direction that surfaced once the first was gone. Because Qwopus
is a thinking model, evaluation ran in non-thinking mode so each judged response is a real answer
rather than an unfinished <think> block; the shipped model restores the original thinking chat
template.
| Round | Refusals | Note |
|---|---|---|
| Baseline | 99/100 | original model |
| Round 1 | 25/100 | dominant refusal direction removed |
| Round 2 | 3/100 | residual direction removed (final) |
- Datasets:
mlabonne/harmless_alpaca(good) vsmlabonne/harmful_behaviors(bad). - Final KL divergence 0.0398 (capability preserved, well under the 0.5 damage line).
Quantization
| Scheme | NVFP4, W4A4 |
| Weight rounding | GPTQ (Hessian-based error compensation), MSE observer |
| Weights | FP4 (E2M1), group_size=16, tensor_group, FP8 (E4M3) group scales, shared across fused layers |
| Activations | FP4, dynamic per-group, FP8 (E4M3) scales |
| Quantized | all language-model Linear layers |
| Kept in bf16 | vision tower (model.visual.*), lm_head, MTP head |
| Untouched | gated delta-net Conv1d and SSM params (A_log, dt_bias), never Linear |
GPTQ is a quantization-time cost only; inference speed and format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.
Calibration: 512 domain-matched samples (long reasoning + general chat + code),
max_seq_len=2048, text-only path through the VL model.
Recommended sampling
Thinking mode is the default.
- Thinking, precise coding:
temperature=0.6,top_p=0.95,top_k=20 - Thinking, general:
temperature=1.0,top_p=0.95,top_k=20 - Instruct / non-thinking:
temperature=0.7,top_p=0.80,top_k=20 - To run non-thinking, set
{%- set enable_thinking = false %}in the chat template, or passextra_body={"chat_template_kwargs": {"enable_thinking": false}}.
Related
- Base model: Jackrong/Qwopus3.6-27B-Coder
- Space: Rogue Quants
- Collection: NVFP4 Quants
- Sibling NVFP4 quants:
Notes
- Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
--reasoning-parserand--tool-call-parserare not auto-detected; pass them explicitly.- Thinking mode is on by default; toggle it via the chat template or
chat_template_kwargs. - No refusal guardrails; you are responsible for how you use it.
License
Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.
Credits
- Base model: Jackrong
- Abliteration: Heretic by Philipp Emanuel Weidmann
- Quantization tooling: llm-compressor / compressed-tensors
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_5
- License
- apache-2.0
- Languages
- en, zh
- Created
- Jul 2, 2026
- Updated
- Aug 4, 2026