ActivationSteeringRepair

Activation Steering Repair

This repository contains the selected Gaussian denoising-autoencoder (DAE) checkpoints from the Activation Steering Repair study. The checkpoints repair Qwen3 residual-stream activations and are not standalone language models.

• Code and reproducibility materials
• IEEE-style report

Checkpoints

Each directory contains denoiser.safetensors, config.json, and metrics.json. The exact base-model and WikiText revisions are recorded in the config files. SHA-256 values are listed in checksums.json.

Training protocol

The residual DAE was trained at the output of transformer block 13 on clean WikiText-2 activations with isotropic Gaussian displacement. Corruption strength is conditioned by

rho = ||noise|| / ||clean activation||.

Training uses no persona vectors, trait labels, extraction prompts, or final evaluation generations. The checkpoint is selected only by reconstruction MSE on a held-out clean-activation split, subject to an identity-MSE constraint.

Loading

Install the project from GitHub, download the matching checkpoint, and load it with the repository implementation:

import torch
from huggingface_hub import hf_hub_download

from steering_research.denoising.model import load_denoiser_checkpoint

path = hf_hub_download(
    repo_id="DikovAlexandr/ActivationSteeringRepair",
    filename="Qwen3-0.6B/denoiser.safetensors",
)
denoiser = load_denoiser_checkpoint(path, device="cuda")

# hidden is the steered block-13 activation; rho is its relative strength.
repaired = denoiser(hidden, rho=torch.tensor(0.5, device="cuda"))

The paper's orthogonal denoising repair (ODR) removes from the DAE correction the component parallel to the steering vector. The complete hook and repair implementations are in the linked GitHub repository.

Results and limitations

ODR preserved the selected steering coordinate to numerical precision, but scaled ODR did not improve the preregistered LLM-judge steering--fluency trade-off over raw activation addition on either Qwen3 model size. These weights should therefore be treated as research artifacts supporting a controlled negative result, not as a generally effective steering repair.

Only two sizes from one model family, one layer, and four persona traits were tested. Evaluation used GPT-5.6 Sol and Claude Opus 4.8 as blinded LLM judges; it was not a human-preference study.