glp-surprise-steering-llama3.2-1b

GLP-derived surprise steering direction for Llama-3.2-1B

This package contains two unit-normalized residual-stream directions:

• raw_direction: the original GoEmotions DiffMean direction;
• selected_direction: the direction selected from an offline Generative Latent Prior trajectory.

They target the residual stream after zero-based decoder layer 7 of unsloth/Llama-3.2-1B / meta-llama/Llama-3.2-1B. The recommended direct steering amplitudes are stored in the safetensors metadata and in selection.json.

Result

On 512 untouched prompts with greedy 20-token continuations, the selected direction was within the preregistered $\pm 0.05$ concept-equivalence margin relative to raw steering and reduced conditional NLL by 0.803:

The candidate used an angle of $43.27^\circ$ from the raw direction and a direct amplitude of 2.122, versus 5.123 for raw steering. It removed about 77% of the extra NLL introduced by the raw intervention.

Loading

from safetensors.torch import load_file

directions = load_file("surprise-steering-directions.safetensors")
raw = directions["raw_direction"]
selected = directions["selected_direction"]

Apply selected_alpha * selected_direction to the residual stream after layer 7. The checkpoint does not contain Llama weights and does not require GLP at inference time.

Scope and limitations

This is a task-specific research artifact, not a universal steering denoiser. The search pipeline can abstain and return the raw vector when confirmation fails. The result was tested on one 1B model, one layer and greedy decoding. Concept strength is operationally defined by the fixed SamLowe/roberta-base-go_emotions classifier on completion-only text.

See the repository REPORT.md for the full method, cross-concept results, negative cases and reproducibility details.

Full report and source code: Tmi-creator/offline-glp-steering.