activation-oracle-legacy-gemma-4-26B-A4B-it

Legacy Activation Oracle: gemma-4-26B-A4B-it

Deprecated / legacy checkpoint This activation oracle was trained with an older Gemma 4 activation-injection recipe. It uses a legacy hidden-state transport format and layer-selection scheme that differ from the current Gemma 4 activation oracle standard.

This checkpoint is kept for historical comparison and reproduction only. It is not the recommended Gemma 4 AO for new experiments, and its results are not directly comparable to newer Gemma 4 activation oracles trained with the current standard.

This is a legacy LoRA adapter for gemma-4-26B-A4B-it, presented in the paper Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals.

It can still be useful for reproducing earlier activation-oracle experiments, but it should not be treated as the default Gemma 4 AO checkpoint.

Why This Checkpoint Is Legacy

This model was trained before the current Gemma 4 AO injection convention was adopted. In practice, that means:

• it uses an older activation transport / injection recipe
• it uses an older layer-selection convention
• it should be treated as a historical artifact rather than the default Gemma 4 AO

Classification-style evaluations may still look reasonable, but that does not make this checkpoint the right choice for current Gemma 4 AO work.

When To Use It

Use this checkpoint only if you specifically want to:

• reproduce earlier Gemma 4 AO results
• compare older and newer AO training conventions
• inspect how the legacy recipe behaves

For new Gemma 4 AO experiments, use a checkpoint trained with the current standard instead.

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-4-26B-A4B-it",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-26B-A4B-it")

model = PeftModel.from_pretrained(base_model, "EvilScript/activation-oracle-legacy-gemma-4-26B-A4B-it")
model.eval()

Legacy Training Details

Related Resources

• Paper: Confidence and Calibration of Activation Oracles (arXiv:2605.26045)
• Code: probabilistic_activation_oracles