qwen3.6-35b-a3b-ao

Qwen3.6-35B-A3B Activation Oracle

An Activation Oracle (AO): a LoRA adapter that makes Qwen3.6-35B-A3B answer arbitrary natural-language questions about activation vectors taken from a language model. You feed it a residual-stream activation; it tells you what that activation contains.

This oracle interprets its own base model's activations (oracle and target are both Qwen3.6-35B-A3B).

Trained following "Building Better Activation Oracles" (arXiv:2606.02609), the follow-up to "Activation Oracles" (arXiv:2512.15674, Karvonen et al.), which builds on LatentQA (arXiv:2412.08686).

How it works

Activations are injected into the oracle's residual stream after decoder layer 1, at placeholder token positions, by norm-matched steering:

h'_i = h_i + steering_coefficient * ‖h_i‖ * (v_i / ‖v_i‖)

The prompt marks injection slots with the single token " ?" (id 907), in one block per source layer:

Layer: 23
 ? ? ? ? ?
Layer: 24
 ? ? ? ? ?
...
Layer: 27
 ? ? ? ? ?
<your question about the activations>

Key configuration (also in ao_config.json):

Collect the activations from the base model with adapters disabled; generate the answer with the adapter active. Activations are the decoder block output (residual stream), bf16, d_model = 2048.

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen3.6-35B-A3B"
LAYERS = [23, 24, 25, 26, 27]
INJECT_LAYER, COEF, PLACEHOLDER = 1, 1.0, " ?"

tok = AutoTokenizer.from_pretrained(BASE)
tok.padding_side = "left"
model = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16,
                                             device_map="auto", attn_implementation="sdpa")
model = PeftModel.from_pretrained(model, "stanleytheli/qwen3.6-35b-a3b-ao")
model.eval()

layers = model.base_model.model.model.layers  # adjust if the wrapper nests language_model

# 1) Collect activations from the BASE model (adapters disabled) at LAYERS,
#    for whichever token positions you care about  ->  vectors: (num_layers * num_positions, 2048)
# 2) Build the prompt: one "Layer: L\n" + PLACEHOLDER*num_positions + " \n" block per layer,
#    then your question; render with tok.apply_chat_template(..., enable_thinking=False).
# 3) Register a forward hook on layers[INJECT_LAYER] that, at the placeholder positions,
#    does: resid[pos] = resid[pos] + COEF * ‖resid[pos]‖ * normalize(vector)
# 4) Generate with the adapter active.

A complete reference implementation of steps 1–4 lives in adamkarvonen/activation_oracles (nl_probes/utils/steering_hooks.py, nl_probes/utils/dataset_utils.py) and its multi-layer fork japhba/activation_oracles.

Training

Trained only on generic interpretability data — no model-organism-specific or behavior-targeted data was used.

Evaluation

Scored with AObench, judged by Qwen3.5-122B-A10B. Compared against a sibling oracle trained with the original (v1) recipe — single-layer feeding, LatentQA SPQA data, LoRA r=64.

activation_sensitivity = 0.911 indicates the oracle's answers depend on the injected activations rather than on the surrounding prompt text.

Classification test accuracy (n=2250 each): geometry_of_truth 0.948, tense 0.962, ner 0.923, snli 0.919, sst2 0.858, md_gender 0.867, relations 0.834; held-out: singular_plural 0.924, ag_news 0.808, language_identification 0.724.

Limitations

• Plausible errors. This oracle rarely hedges (vague-response rate 0.086) but produces plausible-but-wrong answers at a rate of 0.754 on the hallucination eval. Confident phrasing is not evidence of correctness. Verify anything load-bearing.
• System-prompt questions are weak. A v1-recipe oracle scores much higher on system-prompt QA (4.55/4.68 vs 3.64/3.48 judge scores), because that recipe trains on LatentQA system-prompt data, which this recipe drops.
• Layer-specific: use the layers it was trained on (23–27). Other layers are untested.
• Not evaluated on taboo / persona secret-elicitation tasks (those need fine-tuned target organisms).
• Same-model only: trained to read Qwen3.6-35B-A3B activations, not other models'.

Citation

@article{activation_oracles_2025,
  title  = {Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers},
  author = {Karvonen, Adam and Chua, James and Dumas, Cl\'ement and Fraser-Taliente, Kit and
            Kantamneni, Subhash and Minder, Julian and Ong, Euan and Sen Sharma, Arnab and
            Wen, Daniel and Evans, Owain and Marks, Samuel},
  journal = {arXiv preprint arXiv:2512.15674},
  year   = {2025}
}