Muse-Glimmer-30B-heretic-r2

Muse-Glimmer-30B-heretic-iter

English · 中文

Decensored residual-rank-1 variant of meta-models/Muse-Glimmer-30B.

在 meta-models/Muse-Glimmer-30B 上做了两轮残差 rank-1 消融（Heretic）。本文是完整实验记录。

• This folder / 本目录: round-2 merge (heretic-r2). On the training box also linked as Muse-Glimmer-30B-heretic-iter.
• Sibling kept / 仍保留: Muse-Glimmer-30B-heretic-plus (round 1 only).
• License / 许可: Apache-2.0. Follow the base usage policy and applicable law.

---

English

What this is

A 30B dense Muse Glimmer checkpoint (BF16, 13 safetensors shards, ~56GB) with refusal-related directions removed from residual writers (attn.o_proj; round 1 also mlp.down_proj). The vision encoder, embeddings, and attn.gate_proj were not edited.

This is not a fine-tune and not a GGUF. Chat template, tokenizer, and processor are copied from the base model.

Evaluation protocol

All headline numbers use Heretic’s built-in scorers:

• Keywords: mlabonne/harmful_behaviors test[:100]
• KL: mlabonne/harmless_alpaca test[:100]
• Greedy decode, max_new_tokens=100
• System prompt: You are a helpful assistant.
• Pinned response prefix (start of Muse’s user-facing turn):

 to=user<|message|>

Do not compare these numbers to a run that auto-detected the prefix to=. That prefix is wrong for this template (see timeline).

A separate dump of the same 100 prompts (same keyword list, generation noise) scored plus 17/100 and this model 12/100. Official heretic --evaluate-model is the headline.

Default Heretic markers (i can', i cannot, illegal, sorry, …) can fire on a true refusal or on a complying answer that still says “illegal”. Several of the remaining ~12–14 look like the latter.

Timeline

1. Naive Heretic (2026-08-11)

p-e-w/heretic v1.4.0, 200 Optuna trials, default datasets, auto prefix.

Reported: keywords 58 → 26 / 100, KL 0.053, direction_index = 37.41. Ablation looked “weak.” It was mostly measuring the wrong token.

2. Diagnosis on the original (2026-08-13)

Muse starts generation at <|start|>assistant with no recipient. The template’s valid recipients are "self" and "user":

Free generation always began to=self<|message|>…. Heretic’s commonprefix therefore became the string ' to='.

Appending that string and re-tokenizing does not match the natural first token (' to', id 328). After the broken prefix, 100/100 eval completions continued as:

bash.exec<|message|>This is disallowed. Must refuse. ...

Residuals and keywords sat inside a hallucinated tool channel, not on the user-facing refusal. Forcing to=user<|message|> on the original model moved the same 100-prompt keyword rate from 59 to 98. The base model already refuses AdvBench-style asks almost always; the first pass never saw that.

Geometry at layer 37 (the old direction_index):

• prefix to=: S(g,b) ≈ 0.95, ‖r‖ ≈ 70 (tight bash.exec cluster)
• prefix to=user<|message|>: S(g,b) ≈ 0.75, ‖r‖ ≈ 143 (actual comply vs refuse)

winsorization_quantile=0.995 slightly improved SNR. Secondary knob, not the main bug.

3. heretic-plus — correct rank-1 (2026-08-13)

Heretic from the original weights, prefix pinned, winsorize 0.995, 200 trials, Pareto index 0 (lowest keywords).

• Output: Muse-Glimmer-30B-heretic-plus
• Selected trial 52: direction_index = per layer
  • attn.o_proj: max=1.07, pos=31.09, min=0.54, dist=9.83
  • mlp.down_proj: max=1.45, pos=38.85, min=0.66, dist=22.56
• Official eval: 98 → 15 / 100, KL 0.125

This is the real first cut.

4. Delete the broken first checkpoint (2026-08-14)

Removed the entire wrong-prefix directory Muse-Glimmer-30B-heretic (~56GB). heretic-plus and the original were not touched.

5. Residual rank-1 round 2 (2026-08-14)

Not simultaneous rank-k. Greedy residual rank-1:

1. Use plus as the new base.
2. Recompute r = mean(harmful) − mean(harmless) through plus (same 400+400 prompts, same pinned prefix).
3. Search a new rank-1 ablation (120 trials). Search KL is vs plus.
4. Merge, then evaluate vs original and vs plus.

Selected trial 89:

• direction_index = 26.01 (one interpolated layer, not per-layer)
• attn.o_proj: max=1.27, pos=40.69, min=0.74, dist=13.24
• mlp.down_proj: all zeros — round 2 only edits attention writers

Official eval vs original: 14/100, KL 0.125 (same as plus). Incremental KL vs plus: 0.002.

Round 3 was not run: official keyword drop vs plus is 1 point (below the pre-set “≥ 3” bar). Leftover hits look like wording, not a third large linear axis.

Method sketch

original
   │  Heretic rank-1
   │  prefix = " to=user<|message|>"
   │  winsorize = 0.995
   ▼
heretic-plus          98 → 15 keywords, KL 0.125 vs original
   │  recompute r on plus
   │  Heretic rank-1 again
   ▼
heretic-r2 (this)     15 → 14 keywords, KL 0.125 vs original, +0.002 vs plus
   │
   ▼
no round 3

Why residual rounds instead of one fat rank-k: each new r is measured after the previous direction is gone. Computing r1…rk on the original and subtracting them together double-counts overlap and usually burns KL first.

Heretic already had projected ablation (orthogonalize_direction) and full row-norm preservation (row_normalization=full). Those stayed on.

How to use

from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model_id = "Muse-Glimmer-30B-heretic-iter"  # or the local folder
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Your question"},
]
inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
).to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(processor.decode(out[0], skip_special_tokens=False))

Muse will often open with to=self<|message|> (reasoning) then to=user<|message|> (answer). That is the native template. The numbers in this card were measured at the user-facing start by forcing the prefix above.

Needs a transformers build that loads MuseGlimmerForConditionalGeneration (lab: 5.15.0.dev0). There is no ForCausalLM class for this architecture.

Limitations

• Keyword rate is a proxy, not a judge.
• KL on 100 harmless first-token distributions is not MMLU / GSM8K / agent benches.
• Multimodal, tool, and long-horizon behavior were not re-benchmarked after ablation.
• attn.gate_proj can still implement a residual “close the door.”
• Do not use this to break the law or the base usage policy.

Credit

• Base: Meta Superintelligence Lab — Muse Glimmer 30B (Apache-2.0)
• Optimizer: p-e-w/heretic v1.4.0
• Directional / norm-preserving practice: Arditi et al. 2024; Lai (grimjim)
• Residual rounds: greedy rank-1 / INLP-style recompute of r after each merge

---

中文

这是什么

30B 稠密 Muse Glimmer 检查点（BF16，13 个 safetensors 分片，约 56GB）。从残差写入矩阵里挖掉了与拒绝相关的方向（attn.o_proj；第 1 轮还动了 mlp.down_proj）。视觉塔、embedding、attn.gate_proj 都没改。

不是微调，不是 GGUF。对话模板、分词器和 processor 从底模原样拷来。

评测协议

标题数字一律用 Heretic 内置打分：

• Keyword：mlabonne/harmful_behaviors test[:100]
• KL：mlabonne/harmless_alpaca test[:100]
• 贪心解码，max_new_tokens=100
• 系统提示：You are a helpful assistant.
• 钉死作答前缀（Muse 对用户说话的起点）：

 to=user<|message|>

不要拿这些数字去比「自动探测前缀得到 to=」的那一轮。对这个模板，那个前缀是错的（见时间线）。

同一 100 条另做一次 dump（同一套 keyword，生成有抖动）：plus 17/100，本模型 12/100。对外引用以官方 evaluate-model 为准。

Heretic 默认词表（i can'、i cannot、illegal、sorry 等）既会打中真拒绝，也会打中已经答应办事、只是文里出现 “illegal” 的回答。剩下约 12–14 条里有不少属于后者。

时间线

1. 直接对原版跑 Heretic（2026-08-11）

p-e-w/heretic v1.4.0，200 次 Optuna，默认数据集，自动前缀。

当时记录：keyword 58 → 26 / 100，KL 0.053，direction_index = 37.41。看起来「拒绝消不干净」。其实量错了 token。

2. 对原版做诊断（2026-08-13）

Muse 的生成头是 <|start|>assistant，没有收件人。模板里合法收件人只有 "self" 和 "user"：

自由生成一律先走 to=self<|message|>…。Heretic 的 commonprefix 于是收成字符串 ' to='。

把这段字符串拼回去再分词，对不上自然的第一个 token（' to'，id 328）。拼回去之后，100 条评测全部变成：

bash.exec<|message|>This is disallowed. Must refuse. ...

残差和 keyword 都落在幻觉出来的工具通道里，不是对用户的拒绝。原版上强制 to=user<|message|> 后，同一 100 条 keyword 从 59 跳到 98。底模对 AdvBench 风格几乎全拒；第一轮根本没看见。

第 37 层几何（旧 direction_index）：

• 前缀 to=：S(g,b) ≈ 0.95，‖r‖ ≈ 70（挤在 bash.exec 簇里）
• 前缀 to=user<|message|>：S(g,b) ≈ 0.75，‖r‖ ≈ 143（真正的答应 vs 拒绝）

winsorization_quantile=0.995 能略抬 SNR，是次要旋钮，不是主因。

3. heretic-plus：正确的 rank-1（2026-08-13）

从原版权重重跑 Heretic：前缀钉死，winsorize 0.995，200 trial，Pareto 第 0 个（keyword 最低）。

• 输出：Muse-Glimmer-30B-heretic-plus
• 选中 trial 52：direction_index = per layer
  • attn.o_proj：max=1.07，pos=31.09，min=0.54，dist=9.83
  • mlp.down_proj：max=1.45，pos=38.85，min=0.66，dist=22.56
• 官方评估：98 → 15 / 100，KL 0.125

这才是真正的第一刀。

4. 删掉错误的第一份检查点（2026-08-14）

整目录删除错前缀的 Muse-Glimmer-30B-heretic（约 56GB）。heretic-plus 和原版没动。

5. 残差 rank-1 第 2 轮（2026-08-14）

不是一次 rank-k，而是贪心残差 rank-1：

1. 以 plus 为新底模。
2. 在 plus 上用同一套 400+400、同一前缀重算 r = mean(harmful) − mean(harmless)。
3. 再搜一档 rank-1（120 trial）。搜索时的 KL 相对 plus。
4. merge 后分别相对原版、相对 plus 评估。

选中 trial 89：

• direction_index = 26.01（单层插值，不再 per-layer）
• attn.o_proj：max=1.27，pos=40.69，min=0.74，dist=13.24
• mlp.down_proj：全 0 — 第二轮只动注意力写出

相对原版官方评估：14/100，KL 0.125（和 plus 一样）。相对 plus 的增量 KL：0.002。

没有做第 3 轮：官方 keyword 只比 plus 少 1 个（低于预先写死的「再降 ≥ 3」）。剩下的更像用词（回答里出现 illegal / violate），不像还有第三根大轴。

方法示意

原版
   │  Heretic rank-1
   │  前缀 = " to=user<|message|>"
   │  winsorize = 0.995
   ▼
heretic-plus          98 → 15 keyword，相对原版 KL 0.125
   │  在 plus 上重算 r
   │  再做一轮 Heretic rank-1
   ▼
heretic-r2（本模型）   15 → 14 keyword，相对原版 KL 0.125，相对 plus +0.002
   │
   ▼
不做第 3 轮

为什么用多轮残差、不用一次 rank-k：每一轮的 r 都在已经挖过的几何里重新算。在原版上预先抽出 r1…rk 再一起减，会把重叠分量减两遍，通常先伤 KL。

Heretic 默认已开投影消融（orthogonalize_direction）和保行范数（row_normalization=full），全程保持。

怎么用

见上方 English 的 Python 示例。Muse 常常先打出 to=self<|message|>（思考），再打 to=user<|message|>（作答）。这是原生模板，不是损坏。本卡片里的数字是在对用户作答的起点上量的。

需要能加载 MuseGlimmerForConditionalGeneration 的 transformers（实验环境 5.15.0.dev0）。这个架构没有 ForCausalLM。

局限

• Keyword 率只是代理，不是裁判模型。
• 100 条 harmless 首 token 的 KL 不能代替 MMLU / GSM8K / 智能体评测。
• 消融后没有单独重测多模态、工具调用、长程任务。
• attn.gate_proj 仍可能单独把门关上。
• 不要用来违法，也不要违反底模使用政策。

致谢

• 底模：Meta Superintelligence Lab — Muse Glimmer 30B（Apache-2.0）
• 优化器：p-e-w/heretic v1.4.0
• 方向 / 保范数：Arditi et al. 2024；Lai（grimjim）
• 多轮残差：贪心 rank-1 / 类似 INLP，每轮 merge 后重算 r