Qwen3.5-9B-Uncensored

Qwen3.5-9B-Uncensored

We're exploring a new abliteration strategy and used the 9B model as a testbed. The result turned out well — this is currently the state-of-the-art abliterated 9B model across all three axes at once: capability retention, output brevity, and refusal removal. Three variants are published for different deployment needs — pick the one that matches your setup.

我们在尝试一个新的消融策略,以 9B 模型作为练兵场。效果不错 —— 这是目前 abliterated 9B 领域在能力保留、输出简洁、拒绝移除三方面同时达到最优的一版。 下面三个仓分别对应不同 的部署场景,按需选择即可。

This repo is the full-precision bf16 reference weight, 19 GB. Quantized variants: -W4A16 — INT4 weight + FP16 activation, 8.2 GB, for vLLM serving on 12-24 GB GPUs · -GGUF — seven llama.cpp quantization levels from Q3_K_M (~4.5 GB) to BF16 (~18 GB), covering laptops to workstations.

本仓是全精度 bf16 参考权重,19 GB。 量化版本: -W4A16 — INT4 权重 + FP16 激活,8.2 GB,面向 vLLM 部署(12-24 GB GPU)· -GGUF — 七档 llama.cpp 量化(Q3_K_M ~4.5 GB 至 BF16 ~18 GB),覆盖笔记本到工作站。

---

Based on Qwen/Qwen3.5-9B, refusal directions are removed from the residual stream via directional ablation (Arditi et al. 2024). The vision tower is untouched — image understanding is identical to the base model.

在 Qwen/Qwen3.5-9B 基础上,通过 Arditi et al. 2024 方向性消融移除残差流中的拒绝方向。视觉塔零改动, 图像理解与原模型完全一致。

All numbers below are measured, not cited. All compared models (base + two community ablations + this model) were run on the same machine, same harness, same sampled question set, same fixed seed, same decoding settings — comparisons are fair.

以下所有数字都是实测,不是引用。所有对比模型(base + 两个社区消融版 + 本模型)在同 一台机器、同一 harness、同一批采样题、同一固定 seed、同一解码设置下跑,可比性完整。

---

1. Key results / 核心结论

• Refusal rate: 100 held-out harmful prompts, base refuses 99 → this model refuses 2 (effectively 0).

• Capability retained: 11-benchmark mean -0.54pp (77.47 → 76.93) — the best capability retention among the abliterated family.

• Ablation also improved verbosity: median output tokens -17.7%, total generation tokens -0.9%, truncation rate -15% — more concise than base, an order of magnitude more concise than the community ablations.

• 拒绝率:100 条 held-out harmful,base 拒绝 99 条 → 本模型拒绝 2 条(实用上等于 0)

• 能力保留:11 项基准均值 -0.54pp(vs base 77.47 → 76.93)—— abliterated 家族中 能力保留最好的一版

• 消融同时改善过度思考:中位输出 token -17.7%、11 项总输出 -0.9%、截断率 -15% —— vs base 尚且更简洁,vs 社区消融版更是差一个数量级

---

2. Refusal comparison / 拒绝率对比

100 harmful prompts drawn from mlabonne/harmful_behaviors test split, 50 harmless controls from mlabonne/harmless_alpaca. Greedy decoding, 192 tokens, thinking disabled. A fixed judge model reads the full response text to decide refusal (not keyword matching — not fooled by hedging language).

100 条 harmful 提示来自 mlabonne 数据集 test split,50 条 harmless 对照来自同源。 贪心解码,192 tokens,思考关闭。一个固定裁判模型读整段完整回复判定 refuse。

---

3. Capability comparison / 能力评测

Same fixed seed, same harness, same items. All 4 models run under vLLM TP=2 + fp8_e5m2 KV + 24576 max output tokens, thinking enabled.

同一固定 seed,同 harness,同题,4 个模型都在同 vLLM TP=2 + fp8_e5m2 KV + 24576 max output tokens 下跑,开启 thinking。

Among the abliterated family (excluding base): 7 firsts out of 11, mean 76.14 leads Heretic-v2 (72.63, +3.5pp) and Huihui (58.58, +17.6pp) by wide margins.

在 abliterated 家族中(不算 base):7 项第一 / 11,均值 76.14 领先 Heretic-v2 3.5pp, 碾压 Huihui 17.6pp。

† TruthfulQA: when run serially with the other 10 benchmarks, vLLM prefix-cache / KV state contamination pulled the score to 54. Re-run in isolation with the same seed: 62.68. Table uses the fair number.

† TruthfulQA:11 项串行跑时 vLLM prefix cache/KV state 累积导致偏低(54);同 seed 单独重跑 得 62.68(与其它项排除截断分同口径),表内取此数。

TruthfulQA is behavioral drift, not knowledge loss: manual read of the 22 flipped items shows the base prefers "this is a myth" framing while this model prefers the popular claim. Do not use this model as an authority on common-knowledge myths.

不是能力回退,是行为漂移(手工读 22 个翻转题:原模型偏好"这是迷思"式表述,本模型偏好 流行观点)。不要拿本模型当常识性谣言的事实来源。

---

4. Output brevity / 输出简洁度

Qwen's "overthinking" and verbosity are well-known pain points. Same benchmarks, same fixed seed:

Qwen 系列的"过度思考"和"啰嗦"是公认痛点。同基准同固定 seed 下测输出 token 使用:

Fully-ablated community versions expand across the board — Heretic-v2 +66% median, Huihui nearly doubles. This model is the only ablation that is more concise than base (median -17.7%, total slightly below base, truncation -15%). The method removed the refusal direction without breaking the model's "stop when done" capability; over-ablation seems to break that stopping capability.

社区版全线膨胀 —— Heretic-v2 中位 +66%,Huihui 差点翻倍。本模型是唯一比 base 更简洁 的消融版本(中位 -17.7%,总量比 base 略低,截断 -15%)。说明方法只移除了拒绝方向, 没打破 model 的 stop-when-done 能力;过度消融会让 model 停不下来。

---

5. Method / 方法

• Single-direction weight-space ablation (Arditi et al. 2024)

• Direction derived from per-sample harmful-vs-harmless activation differences (not mean difference — more noise-resistant)

• Projection is norm-preserving (each row re-scaled to original L2 after subtracting the projection)

• 67 tensors modified across all layers: linear_attn.out_proj + self_attn.o_proj + mlp.down_proj + embed_tokens + 2 MTP head tensors

• Vision tower and lm_head are untouched

• 权重空间单方向消融,方向从逐样本 harmful vs harmless 激活差取(不是均值差,更抗噪)

• 投影为 norm-preserving(每行去投影后按 L2 缩回原模长)

• 修改 67 个张量:所有层的 linear_attn.out_proj + self_attn.o_proj + mlp.down_proj

  • embed_tokens + 2 个 MTP head 张量;视觉塔和 lm_head 完全未改动

Activation extraction used 128+128 harmful/harmless prompts (mlabonne datasets, chat template applied, thinking off). Specific direction-search, alpha-sweep, and layer-scan hyperparameters are not publicly released (reasonable commercial protection).

激活提取用 128+128 条 harmful/harmless 提示(mlabonne 数据集,应用 chat 模板,关闭思考)。 方向搜索、α 扫描、层扫等具体超参未公开(合理商业保护)。

---

6. Quantized variants and deployment / 量化版本与部署

6.1 vLLM with fp8 KV cache / vLLM + fp8 KV cache

Recommended vLLM launch: 推荐 vLLM 启动:

vllm serve <repo-or-path> \
  --dtype float16 --tensor-parallel-size 2 \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 28672 --max-num-seqs 16 \
  --reasoning-parser qwen3

Deployment case, measured on our test rig (2× RTX 3090; this is what we happen to run on, not a hardware requirement — the model runs on any GPU that fits the weights):

部署案例(这是我们自己的测试机型 2× RTX 3090,非硬件要求 —— 只要显存装得下权重,任何 GPU 都能跑):

• fp8_e5m2 KV vs fp16 KV, KV cache capacity 1.80× (TP=2: 680k → 1.22M tokens)

• Single 3090 with fp8_e5m2 KV also gets 1.67× headroom

• Benchmark accuracy diff in noise range (3 tasks × 450 items, abs delta +0.22)

• fp8_e5m2 KV vs fp16 KV,KV cache 容量 1.80x(TP=2 下从 680k tokens → 1.22M tokens)

• 单张 3090 上 fp8_e5m2 KV 亦有 1.67x 提升

• 各基准精度差在噪声范围(3 项 450 题 abs 差 +0.22)

6.2 MTP speculative decoding / MTP 投机解码

Not recommended. vLLM 0.20.2 shows num_speculative_tokens=1..4 cuts throughput by 40-46%; K=5 triggers CUDA illegal memory. Root cause is architectural (1-layer MTP does multiple forwards per step), not specific to this ablation. Not tested on llama.cpp side.

不推荐启用。vLLM 0.20.2 实测 num_speculative_tokens=1..4 让生成速度下降 40-46%, K=5 触发 CUDA illegal memory。原因是架构性的(1 层 MTP 每步 forward 多次),不是本次 消融特有。llama.cpp 侧未测。

---

7. Known limitations / 已知缺陷

1. Capability cost ≈ mean -0.54pp (11 tasks). Largest per-task drops: MATH-500 -5, HumanEval -4.3, TruthfulQA -24 (behavioral, see below).

2. TruthfulQA is behavioral drift, not knowledge loss. The model has not forgotten correct answers, only stopped preferring "myth-busting" framing. Do not use it as an authority on common-knowledge myths.

3. Refusal removal is not absolute: 2 of 100 held-out harmful still refused. Coverage is English prompts + 2 datasets only. Other languages, other prompt distributions, multi-turn adversarial not tested.

4. MTP speculative decoding unusable on vLLM 0.20.2 (see §6.2).

5. Vision tower is untouched — image understanding inherits base behavior as-is, good or bad. No multi-modal safety re-alignment.

6. 能力代价 ≈ 均值 -0.54pp(11 项)。分项最大掉幅 MATH-500 -5、HumanEval -4.3、 TruthfulQA -24(行为性)

7. TruthfulQA 是行为漂移不是知识损伤,模型没有忘掉正确答案。不要拿它当常识性谣言 的事实来源

8. 拒绝移除不是绝对:100 条 held-out harmful 里仍有 2 条被拒;覆盖范围仅英文提示 + 两个数据集

9. MTP speculative decoding 在 vLLM 0.20.2 上不可用(见 §6.2)

10. 视觉塔未改动,图像理解沿用原模型;多模态输入未做安全再对齐

---

① Provenance and attribution / 来源与归属

Not an official Qwen release; not endorsed by or affiliated with the Qwen team or Alibaba Cloud. "Qwen3.5" identifies the upstream model this artifact derives from (Apache-2.0 §6).

非 Qwen 官方发布;与 Qwen 团队及阿里云无背书或从属关系。"Qwen3.5" 仅用于标识本产物所派生 的上游模型(Apache-2.0 §6)。

---

② Safety alignment has been removed / 安全对齐已被移除

This is the point of the model, and you should read this before downloading. 这是模型的核心属性,下载前请务必阅读。

The ablation removes the refusal behaviour trained into Qwen/Qwen3.5-9B. Automated refusal testing over 100 held-out adversarial prompts recorded 2/100, versus 99/100 for the base model.

本模型的消融移除了 Qwen/Qwen3.5-9B 训练进去的拒绝行为。在 100 条 held-out 对抗提示上 的自动测试记录到 2/100,而基座模型是 99/100。

Consequences you are accepting / 你正在接受的后果:

• It will produce content the original model declines to produce, including content that is offensive, dangerous, or illegal in your jurisdiction. 会产生原模型拒绝产生的内容,包括在你所在司法辖区内可能属于冒犯性、危险或违法的内容
• It has no content filter. There is no safe-completion path, no refusal fallback, no guardrail to fail back to. 没有内容过滤;没有 safe-completion 路径,没有拒绝回落,没有护栏可依
• Refusal removal is not proven exhaustive — absence of observed refusals in testing is not proof that none remain, and equally not proof that no harmful behaviour was introduced. 拒绝移除未证明穷尽 —— 测试中未观察到拒绝不等于证明不存在残留,也不等于证明没有引入 有害行为
• TruthfulQA drops 24 pp vs base. "Do not refuse harmful requests" and "prefer the truthful answer over a plausible-sounding one" are partly aligned optimization targets; pulling on one moves the other. TruthfulQA 相对基座掉 24 pp。"不要拒绝有害请求"和"偏好真实答案而非听起来合理的 答案"是部分对齐的优化目标,拉一个会动另一个

Not intended for / 不适用于:

• Deployment to third parties, end users, or any public-facing service without your own safety layer / 未搭建自己的安全层就部署给第三方、终端用户或任何面向公众的服务
• Anyone under the legal age in their jurisdiction / 未达所在司法辖区法定年龄的人
• Any use prohibited by Qwen's acceptable use policy, which applies to this derivative exactly as it does to the base model / Qwen 可接受使用政策禁止的任何用途(该政策适用于 本派生物,与基座模型一致)

Intended for / 适用于: local inference and research, by people who understand the above and take responsibility for it / 本地推理与研究,由理解以上内容并为其负责的用户使用。

---

③ No warranty; responsibility rests with the user / 无担保;责任在用户

This model is provided "AS IS", without warranty of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, and non-infringement.

本模型按**"现状"提供,不提供任何明示或默示的担保**,包括但不限于对适销性、特定用途 适用性以及无侵权性的担保。

• I do not endorse, recommend, or condone any particular use of this model. 我不背书、不推荐、不认可对本模型的任何特定使用方式
• I make no representation that its outputs are accurate, lawful, or fit for any purpose. 我不对其输出的准确性、合法性或用途适用性作任何声明
• You are solely responsible for what you generate with it, for how you deploy it, and for compliance with all laws and regulations applicable to you — including but not limited to laws on illegal content, data protection, export control, and AI-specific regulation in your jurisdiction. 你独自承担全部责任:对生成内容、部署方式、以及遵守适用于你的一切法律法规负责 —— 包括但不限于关于非法内容、数据保护、出口管制、以及所在司法辖区 AI 专项监管的法律
• To the maximum extent permitted by law, I accept no liability for any claim, damage, or other liability arising from the model or its use. 在法律允许的最大范围内,我不承担因本模型或其使用而产生的任何索赔、损害或其他责任

Downloading these files means you accept the above. If you do not, do not download them. 下载本仓文件即表示你接受以上内容。若不接受,请勿下载。

The author is not a lawyer; the above is not legal advice. 作者不是律师;以上不构成法律意见。

---

Attribution / 简署名

Base model, tokenizer, chat template, vision tower, and all capabilities are inherited from Qwen, Apache 2.0. This ablation is a mechanical weight-space edit on top; no additional training data was used.

基座模型、tokenizer、chat 模板、视觉塔、所有能力均继承自 Qwen,Apache 2.0。本次消融是叠加在其上的 一次机械式权重编辑,未使用任何额外训练数据。

License

Apache 2.0, inherited through the chain above. The Apache-2.0 grant covers the weights; it does not grant permission for uses that are unlawful where you are.

Apache 2.0,依上述归属链继承。Apache-2.0 授权覆盖权重本身;不授权任何在你所在地不合法 的用途。

Citation / 引用

Arditi et al. Refusal in Language Models Is Mediated by a Single Direction. 2024.
https://arxiv.org/abs/2406.11717

---

Appendix: summary vs community ablations / 附:与社区消融版对比小结

Only this model achieves all three: low refusal + high capability retention + concise output.

本模型是唯一"低拒绝 + 高能力保留 + 输出简洁"三者兼得的。