Qwen3.8-27B-heretic

Qwen3.8-27B-heretic

Heretic v1.4.0 abliteration of official Qwen/Qwen3.8-27B (1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0).

This is not a new pretrain and not an SFT. Only a subset of language-layer projection weights is modified. Tokenizer, chat template, vision tower, and MTP weights are unchanged.

Vision and MTP are present and loadable, but they were not abliterated.

---

Method

Heretic applies parameterized directional ablation (Arditi et al., 2024): it estimates a refusal direction from first-token residuals, then orthogonalizes selected projections against that direction.

On this model the hooked modules are:

• attention output: attn.o_proj on Gated Attention layers, linear_attn.out_proj on Gated DeltaNet layers (Heretic maps both to attn.o_proj)
• MLP: mlp.down_proj on all 64 language layers

That is 64 attention-output modules and 64 MLP modules. Embeddings, norms, other attention/MLP projections, the vision encoder, and the MTP block are left as in the base checkpoint.

TPE search (200 trials, 60 startup) jointly minimized English keyword refusals on mlabonne/harmful_behaviors and KL divergence on mlabonne/harmless_alpaca. Trial 145 (Pareto index 0, lowest keyword count) was merged to a full BF16 checkpoint.

Instruct cut point

Qwen3.8 thinks by default. apply_chat_template(..., add_generation_prompt=True) starts the assistant turn with an open <think>\n. Official instruct / non-thinking mode closes that as:

<think>

</think>

Heretic’s default CoT skip looks for exact <think></think>, which this model does not emit. Residuals and KL collected at the open <think> token are dominated by the thinking opener, not by the answer distribution.

Search and evaluation therefore used:

--response-prefix $'\n</think>\n\n'

This is only a measurement cut. The shipped chat_template.jinja is the official one. At inference time, use enable_thinking=False for instruct mode; do not bake the Heretic prefix into the template.

Selected parameters (trial 145)

Reproduce

PREFIX=$'\n</think>\n\n'

heretic \
  --model Qwen/Qwen3.8-27B \
  --response-prefix "$PREFIX" \
  --n-trials 200 \
  --n-startup-trials 60 \
  --winsorization-quantile 1.0 \
  --model-action save \
  --export-strategy merge \
  --save-directory Qwen3.8-27B-heretic

heretic \
  --model Qwen/Qwen3.8-27B \
  --evaluate-model Qwen3.8-27B-heretic \
  --response-prefix "$PREFIX"

---

Evaluation

Same instruct cut as above. Keyword markers and both prompt sets are English.

Sanity checks on the merged weights (greedy, short decode): instruct 1+1 answers 2 directly; a Fibonacci request yields a normal Python function; thinking mode still emits <think>…</think> then answers.

Abliteration lowers templated English refusals. It is not a safety guarantee. Chinese refusal rate and vision/MTP quality were not measured. Thinking-mode refusal behavior can differ from the instruct-cut numbers.

---

Usage

from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "Qwen3.8-27B-heretic"  # replace with the Hub repo id after upload
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain residual connections in one paragraph."},
]

# Instruct / non-thinking (the Heretic evaluation cut)
text = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,
)

# Thinking mode (default): omit enable_thinking, or set True

Recommended sampling follows the base card:

• Thinking: temperature=1.0, top_p=0.95, top_k=20
• Instruct: temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5

Full BF16 needs about 51 GiB of model weights, plus KV cache. For serving, the same stacks as the base model apply (Transformers, vLLM, SGLang, and so on). For architecture details, long-context YaRN, and multimodal serving, see Qwen/Qwen3.8-27B.

---

License

Apache-2.0, inherited from the base model.

@misc{qwen38,
    title = {{Qwen3.8-Max}: A New Bar for Coding and Cowork},
    url = {https://qwen.ai/blog?id=qwen3.8},
    author = {{Qwen Team}},
    month = {August},
    year = {2026}
}

@misc{heretic,
  author = {Weidmann, Philipp Emanuel},
  title = {Heretic: Fully automatic censorship removal for language models},
  year = {2025},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/p-e-w/heretic}}
}

---

中文

官方 Qwen/Qwen3.8-27B 的 Heretic v1.4.0 消融版。不是新预训练，也不是 SFT。只改语言层一部分投影权重；分词器、对话模板、视觉塔和 MTP 与底模相同。

做法

Heretic 用有害 / 无害题的首 token 残差估计拒绝方向，再对选定投影做正交化。本模型挂钩的是语言层：

• 注意力出口：Gated Attention 的 attn.o_proj，Gated DeltaNet 的 linear_attn.out_proj（Heretic 都映射成 attn.o_proj）
• MLP：全部 64 层的 mlp.down_proj

视觉塔和 MTP 未消融。搜索为 200 trial / 60 startup 的 TPE，在 instruct（关思考）切点上同时压 Keyword 拒绝率和无害题 KL。导出的是 Pareto 上 keyword 最低的 trial 145。

为何要钉 instruct 切点

Qwen3.8 默认开思考。apply_chat_template 会在 assistant 开头留下未闭合的 <think>\n。官方关思考空块是 <think>\n\n</think>\n\n。Heretic 默认 CoT skip 找的是精确 <think></think>，对不上。若在未闭合的 <think> 处采残差 / 算 KL，下一个 token 几乎被思考开场锁死，指标会失真。

因此搜索和复评都加了 --response-prefix $'\n</think>\n\n'。这只用于测量，成品模板仍是官方模板。推理时关思考请传 enable_thinking=False。

消融参数见上文英文表。复现命令见上文 Reproduce。

评估

取点均为上述 instruct 切点。Keyword 词表和评估集都是英文。

消融降低的是模板化英文拒答，不是安全保证。中文拒答、视觉和 MTP 本轮未测；开思考时的拒绝行为可能与上表不同。

用法与采样参数见英文 Usage。底模能力、长上下文 YaRN 和多模态部署见官方模型卡。许可为 Apache-2.0。