steering-denoisers

Learned activation priors for steering GPT-2 small

Model weights for the study Cheap learned priors for activation steering.

Code, protocols and the full report: https://github.com/zxclwnq/steering-denoisers

Headline result

A learned activation prior does not provide useful orthogonal naturalization of a steering intervention. Each model below learns its own training task well, and none of them beats a plain hard clamp on language-model quality once the semantic coordinate is held fixed.

The decisive comparison is the constraint-preserving tangent flow:

A hard clamp costs only +0.003 to +0.054 nats while satisfying the coordinate exactly.

Contents

Every directory holds model.pt (a training checkpoint containing model state dict plus optimizer/step metadata), best.json (the selected step and its validation loss) and training_status.json (the completed-run record).

tangent-flow-16m-linear/model.pt is the same checkpoint published separately as qweclownq/tangent-flow-16m, which additionally carries the T1/T2 evaluation receipts.

Where the activations come from

All models operate on the GPT-2 small residual stream at layer 7 (blocks.7.hook_resid_post), read through TransformerLens. from_pretrained applies fold_ln and center_writing_weights, so this basis is numerically not the input to transformer.h[7] in plain HuggingFace transformers. Reproducing the numbers outside TransformerLens requires reproducing that transform first.

Inputs are standardized as x = (h - mu) / sigma with statistics from the training activation set. Flow models predict the velocity u = eps - x_0 along x_t = (1 - t) x_0 + t eps.

Loading

import torch
ckpt = torch.load("tangent-flow-16m-linear/model.pt", map_location="cpu")
state = ckpt["model"]

The matching architecture definition and inference code are in the GitHub repository; these files are raw training checkpoints, not transformers-compatible models.

Training data

Activations collected from FineWeb text. Steering directions come from jbloom/GPT2-Small-SAEs-Reformatted. Held-out directions were never accessed: every artifact of the project records held_out_accessed: false.

Citation

See the repository for the report and the frozen protocol documents behind each number.