Downloads · 30 days
0
pocharlies/qwen38-27b-uncensored-abliterated-refusal-directions
qwen38-27b-uncensored-abliterated-refusal-directions is a machine learning model from pocharlies. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for safetensors. The card lists the license as apache-2.0.
A censored model that uncensors itself when you ask it to — and nothing else changes.
Downloads · 30 days
0
Access
Public
Updated Aug 29, 2026
Repo size
2.6 MB
Likes
8
Public
Click a slice to open those files.
.safetensors2.6 MB · 94%
From the Hugging Face model README
A censored model that uncensors itself when you ask it to — and nothing else changes.
No second 23 GB checkpoint. No baked-in abliterated weights. This is a 2.6 MB pack of direction vectors that plugs into a running vLLM and lets you dial the model's refusal behaviour up or down at runtime, per request, with zero restart:
Two runtimes, one pack of directions. The dial ships for vLLM 0.27.1 and, since 2026-08-29, for SGLang + DSpark — the public DGX Spark recipe. Same vectors, same per-module coefficients, same
cache_salt: "refusal:<x>"seal. Measured on the same shared GB10, single stream, T=0: code 26.0 → 56.1 tok/s (2.2×), prose 21.3 → 18.0. Ablation cost on SGLang: zero on code (56.2 vs 56.0), −4% on prose. Port and caveats:runtime/sglang-dspark/.
| you ask | the model behaves as |
|---|---|
λ=0 (default) | the original Qwen/Qwen3.8-27B — bit-exact, censored |
λ=1 | the abliterated profile — refusals drop 32/60 → 0/60 (StrongREJECT) |
The same weights, the same pod, even the same batch. One HTTP field switches it.
# server-wide dial
curl -XPOST localhost:8888/admin/refusal_lambda -d '{"lambda": 1}' # ablation on
curl -XPOST localhost:8888/admin/refusal_lambda -d '{"lambda": 0}' # off, bit-exact base
Since 2026-08-19 λ can also be set per request, so one pod serves a normal alias and an ablated one from the same weights, in the same batch:
curl -XPOST localhost:8888/v1/chat/completions \
-d '{"model":"qwen38-27b","messages":[...],"cache_salt":"refusal:1.0"}'
Requests without a salt keep using the global dial. This was documented as dead until now:
the obvious implementation is silently wrong, because CUDA graph capture bakes in the global
scalar and replay runs no Python — every graph-served decode used the global λ with no
warning. The fix binds a per-role buffer to the module at construction, which also keeps it
compatible with Dynamo's fullgraph region (measured: compiles once, frames=1, and
mutating the buffer does not recompile). Details and the failing-capable tests are in the
GitHub README.
Code, patch and benchmarks: https://github.com/pocharlies/qwen38-27b-rank1-refusal-projection
The exact ARM64 compatibility port is published in the GitHub repository under
runtime/vllm-0.27.1/.
The Unsloth NVFP4 + native MTP k=3 configuration was not changed. It passed health,
tooling and refusal-dial checks, but was slightly slower than 0.25.2: 26.01 vs 28.19 tok/s
on code and 21.33 vs 21.72 tok/s on the varied workload. Machine-readable results are in
benchmarks/2026-08-22/vllm-0.27.1.json.
Read before downloading. This material removes or weakens a model's built-in refusal behaviour on demand. It can make a model answer requests it would otherwise decline — some of them clearly harmful.
/admin/refusal_lambda off any public ingress.refusal_dirs_qwen38.safetensors — 128 tensors of F32[5120], plus a __coefs__ tensor and a
coef_order entry in __metadata__ that pairs them.
| module kind | count |
|---|---|
linear_attn.out_proj | 48 |
self_attn.o_proj | 16 |
mlp.down_proj | 64 |
Layers 0–63 complete — every sublayer that writes to the residual stream.
The hook is
y ← y − λ · coef_m · r̂ (r̂ · y)
applied to each sublayer output. coef_m is that module's measured λ_eff, so λ=1 reproduces
the source ablation profile exactly and λ=0 is the untouched base. The per-module coefficient
is not cosmetic: λ_eff ranges 0.999–1.291, and collapsing it to a single mean would introduce
up to 21.8 % error on some modules.
SVD of ΔW = W_abl − W_base in float64, per module, between two publicly released checkpoints:
Qwen/Qwen3.8-27BZynerji/Ektome-Qwen3.8-27B-PristinelyUncensoredBoth BF16, so there is no requantization noise in the difference.
| gate | value |
|---|---|
rank-1 energy s₀²/Σs² | 0.9867 – 0.9927 |
s₀/s₁ | 32.4 – 77.1 |
cos(v₀, u₀ᵀW_base) | 0.936 – 0.9999 (median 0.99987) |
| edit subtracts | 128/128 |
λ_eff | 0.999 – 1.291 |
| projection vs edited weight (float64) | 7.9e-16 |
λ_eff ≈ 1 matters: this source removes the direction exactly once. Baked abliterations often
overshoot — the DeepSeek-V4 reference checkpoint sits at λ_eff ≈ 2.43, i.e. it inverts the
direction, which measurably costs speculative-decoding acceptance.
| candidate | rank-1 energy | s₀/s₁ | cos |
|---|---|---|---|
trohrbaugh/Qwen3.8-27B-heretic-ara | 0.14 – 0.87 | 1.14 – 4.59 | 0.17 – 0.46 |
orwelian84/Qwen3.8-27B-OBLITERATUS-Advanced | 0.32 – 0.55 | 1.19 – 1.90 | 0.9997 |
An s₀/s₁ near 1 means no dominant direction exists. "Abliterated" in a repo name does not
make the edit rank-1 — measure before you build on it.
Same pod, same prompts, temperature 0, seconds between arms:
| λ=0 | λ=1 | |
|---|---|---|
| Refusal rate (5 low-harm triggers) | 5/5 | 0/5 |
| Benign control falsely refused | 0/1 | 0/1 |
| Tool-calling | OK | OK |
| Throughput, alternated (tok/s) | 20.2 / 20.4 | 20.6 / 20.3 |
| MTP acceptance length | 2.80 median (62 samples), max 4.00 at k=3 |
The dial costs nothing measurable in throughput on benign work. MTP acceptance is a more nuanced story — see below.
Do not read ~20 tok/s as this model's speed on a DGX Spark. That figure comes from a node
whose GB10 is shared by five workloads — this model (33 GB), a vision model (30 GB), two BGE
pools and a TTS — hence gpu_memory_utilization: 0.35. The λ=0 vs λ=1 comparison holds,
because both arms ran under identical contention seconds apart; only the absolute number is
depressed. On a dedicated Spark, an independent user running this exact recipe reported
~40 tok/s (third-party report, not measured here).
The deployment smoke above has now been followed by a paired evaluation at λ=0, 1, 1.5 and 2.5: 60 StrongREJECT prompts per arm, 100 GSM8K items, 112 balanced MMLU-Pro items, MTP/speed prompt families and an eight-case tool battery.
| metric | λ=0 | λ=1 | λ=1.5 | λ=2.5 |
|---|---|---|---|---|
| Strict classifier refusals | 32/60 | 0/60 | 0/60 | 29/60 |
| Normal answers / disclaimers | 7 / 21 | 56 / 4 | 60 / 0 | 11 / 20 |
| GSM8K (n=100) | 84% | 81% | 76% | invalid: runaway |
| MMLU-Pro (n=112) | 76.79% | 75.00% | 50.00% | invalid: runaway |
| MTP acceptance, code / varied | 78.01 / 57.93% | 75.64 / 56.89% | 69.04 / 55.40% | invalid |
| Tool battery | 7/8 | 7/8 | 6/8 | invalid |
Against λ=0, the λ=1 quality differences are not statistically detectable in these samples
(GSM8K p=0.25, MMLU-Pro p=0.7905). λ=1.5 significantly degrades both suites. At λ=2.5,
refusal returns almost to baseline and generation becomes pathologically slow. Abliteration
strength is not monotonic after the calibrated edit has been applied once.
Operating point: λ=1. Full methodology, Wilson intervals, paired tests, early-stop records,
machine-readable aggregates and the controlled interpretation of Goldhub's Reddit results are
in benchmarks/2026-08-17/.
Measured on the served pod, 300 tokens, temperature 0:
| prompt | λ | MTP acceptance |
|---|---|---|
| benign (write quicksort) | 0 | 2.84 |
| benign (write quicksort) | 1 | 3.00 |
| refusal trigger (phishing email) | 1 | 2.41 |
Acceptance holds on benign work but drops ~20% on exactly the topics someone turns the dial on for. This was predicted before it was measured: "the patch only touches the main model, not the drafter, so on rejected topics the mini-model keeps refusing and acceptance collapses — the patch has to change the drafter's lambda too."
The effect is real. The proposed cause is not. Projecting the drafter with a backbone
direction (VLLM_REFUSAL_MTP_MODE=mean) does not recover it — measured, 3 reps per cell:
| refusal λ=1 | benign λ=1 | gap | |
|---|---|---|---|
| drafter unprojected | 2.41 | 3.00 | 0.59 |
| drafter projected | 2.43 / 2.54 / 2.44 → 2.47 | 2.77 / 3.16 / 3.14 → 3.02 | 0.55 |
0.59 → 0.55 is noise: the benign column alone spans 0.39 across reps.
The control that settles it: at λ=0 the projection is multiplied by zero, so mean and
off must be identical. They measured 2.94 and 2.84. That 0.10 is the bench's noise floor,
and every "improvement" visible at n=1 sat below it. Do not trust a single sample here.
Best remaining explanation: the backbone already hands the drafter an ablated hidden state, so little refusal signal survives into the drafter's own layer, and the gap is about how hard that content is to predict rather than about who is ablated.
off is the default: it reproduces the source ablation exactly and mean buys nothing
measurable.
Sweeping λ on the live dial (no restart — that is what the dial is for), 4 triggers each, acceptance measured on a long generation for the same refusal topic:
| λ | refusals | acceptance |
|---|---|---|
| 0.3 | 4/4 | 2.72 |
| 0.5 | 3/4 | 2.87 |
| 0.7 | 1/4 | 2.41 |
| 1.0 | 0/4 | 2.61 |
Acceptance is highest at the λ where the model still refuses, and drops once it starts complying. A refusal is formulaic text the drafter predicts easily; the content produced by complying is novel and it does not. Acceptance tracks what is being generated, not λ — note λ=1.0 scores above λ=0.7, which rules out a monotonic penalty in λ.
So there is no λ that both removes refusal and keeps benign-level acceptance, because the drop is the cost of generating the non-refusal content. Nothing to fix here.
Useful by-product: the dose-response on refusal is clean — 4/4 → 3/4 → 1/4 → 0/4 — and λ=1.0 is the operating point. No reason to go lower, and unlike the DeepSeek-V4 reference (which needed 1.5 and sat at λ_eff 2.43), no reason to go higher.
All 128 extracted directions are effectively the same vector — cos against layer 63 is
0.9996 minimum, 1.0000 median, and cos(mean, layer63) = 1.0000. Refusal is mediated by
a single global direction in the residual stream, measured here on a 64-layer hybrid with
linear and full attention interleaved. It also makes last and mean the same choice.
mtp.* tensors, but they are
byte-identical to the base (mtp.layers.0.self_attn.o_proj sha256 9165a16183…), so
there is no direction to extract from them. Acceptance stays high anyway. (Corrected: this
line previously said the tensors were absent — presence is not ablation.)model.visual.* tensors byte-identical
between base and ablation, including merger.linear_fc2.weight, the only one whose output
reaches the residual stream. The gap is zero, not merely small.λ=0 is bit-exact in output but not free in compute: the dot product and subtraction run
in all 128 modules on every token. For zero cost, unset VLLM_REFUSAL_DIRS.A sparkrun recipe ships next to these vectors —
qwen38-27b-nvfp4-refusal-dial.yaml, the exact settings every number above was measured with:
sparkrun run qwen38-27b-nvfp4-refusal-dial.yaml
curl -XPOST localhost:8101/admin/refusal_lambda -d '{"lambda": 1}' # ablation on
curl -XPOST localhost:8101/admin/refusal_lambda -d '{"lambda": 0}' # off, bit-exact
It pulls the stock unsloth/Qwen3.8-27B-NVFP4 at model_revision: main — no second
checkpoint anywhere. Set container: to your own build of the Dockerfile in the GitHub repo
(COPY-only over an existing vLLM image, so it cross-builds to arm64 from x86 without QEMU).
Note on the revision: Unsloth super-squashes their repos, so old pinned SHAs go 404. Don't pin a sha —
mainis the only revision guaranteed to still exist tomorrow.
Two lines in it are load-bearing: pre_exec runs the fail-closed guard before serving, and
--attention-backend triton_attn is not optional on vLLM 0.25.2 — see below.
The cleanest way to use uncensored on demand in a real deployment is a LiteLLM proxy
in front of the vLLM pod. Register the model twice under two aliases that point to the
same backend — the only difference is an extra_body.cache_salt that switches λ per request:
model_list:
- model_name: qwen38-27b # censored profile (default behaviour)
litellm_params:
model: openai/qwen38-27b
api_base: http://vllm-qwen38.llm.svc.cluster.local:8000/v1 # your vLLM pod
- model_name: qwen38-27b-uncensored # uncensored on demand
litellm_params:
model: openai/qwen38-27b
api_base: http://vllm-qwen38.llm.svc.cluster.local:8000/v1 # same pod!
extra_body: { cache_salt: "refusal:1.0" }
qwen38-27b (no cache_salt) → vLLM uses the global dial, which boots at λ=0 → the
original censored model. Bit-exact.qwen38-27b-uncensored (sends cache_salt: refusal:1.0 per request) → vLLM expands that
into a per-token λ → the abliterated profile.model: field.Cost and security of doing it this way:
cache_salt enters the prefix-cache block key, so the two aliases keep separate KV cache
spaces. That lowers hit rate slightly — and it is the correct behaviour: you do not want a
censored request to reuse a block computed under an uncensored λ.team/key model access), so only the
callers you trust can reach qwen38-27b-uncensored. This is how production isolates it from
write-capable tools.Verified in production on this exact pattern (same Service,
extra_body.cache_salt, per-request λ surviving LiteLLM and CUDA-graph replay). The DeepSeek-V4 twin uses the same mechanism atrefusal:1.5.
Requires the vLLM patch in the GitHub repo. Serve the clean checkpoint (or a clean
quantization of it — this was validated against unsloth/Qwen3.8-27B-NVFP4) with
VLLM_REFUSAL_DIRS=/path/refusal_dirs_qwen38.safetensors
VLLM_REFUSAL_LAMBDA_INIT=0.0
--attention-backend triton_attn
flashinfer does not serve Qwen3.5-architecture models on vLLM 0.25.2: it starts fine and fails
on the first real inference with plan(): Mismatched number of arguments.
Reducing a model's resistance to instructions reduces its resistance to injected instructions arriving inside untrusted content. Prompt injection and refusal route through overlapping machinery.
/admin/refusal_lambda off any public ingress.@misc{arditi2024refusal,
title = {Refusal in Language Models Is Mediated by a Single Direction},
author = {Arditi, Andy and Obeso, Oscar and Syed, Aaquib and Paleka, Daniel and
Panickssery, Nina and Gurnee, Wes and Nanda, Neel},
year = {2024},
note = {NeurIPS 2024},
eprint = {2406.11717},
archivePrefix = {arXiv}
}
License: Apache-2.0 for the extraction code. The vectors are derived from the difference between two publicly released checkpoints and inherit the Qwen license.