glp-llama1b-d12-multi

Llama1B Activation Diffusion Model

This repository contains model weights accompanying the paper Learning a Generative Meta-Model of LLM Activations.

• Project page: https://generative-latent-prior.github.io
• Code: https://github.com/g-luo/generative_latent_prior

Quick Start

This model is trained on Llama-3.2-1B activations from all layers (Layer 00-15), using FineWeb data. GLPs are activation diffusion models useful for applications like on-manifold steering and sparse probing.

from glp.denoiser import load_glp

model = load_glp("generative-latent-prior/glp-llama1b-d12-multi", device="cuda:0", checkpoint="final")

Citation

@article{luo2026glp,
  title={Learning a Generative Meta-Model of LLM Activations},
  author={Grace Luo and Jiahai Feng and Trevor Darrell and Alec Radford and Jacob Steinhardt},
  journal={arXiv preprint arXiv:2602.06964},
  year={2026}
}