Boltz1-SAEs-L2-Diffusion

Boltz-1 SAEs — diffusion module

TopK sparse autoencoders trained on activations from Boltz-1's diffusion coordinate module, sampled at three points along the denoising trajectory. Published for anonymous double-blind review; no authorship or affiliation is attached to this repository.

Architecture

Note the input width: 768 here, against 384 for the trunk repositories.

Contents

60 runs across three sampling steps, seven DiffusionTransformer layers and three seeds. Coverage is not a full grid:

Layer 10 is absent at step 10; the other two steps have it.

rec<STEP>/layer<L>/
  diffusion<L>_topk256_lat2048_demean_longtrain500000_l2_3e-3_alive_cross_seed.json
  diffusion<L>_topk256_lat2048_demean_longtrain500000_l2_3e-3_seed<S>/
      checkpoint_step_500000.pt
      config.json
      mean_vector.npy
      eval_step_500000.json
      stats.jsonl

The rec<STEP> directory is the denoising step, not a recycle index. Each config.json records it in its rec field, which the trunk repositories use for the recycle iteration instead.

This extra top-level level is the one structural difference from the trunk repositories, whose runs sit directly under layer<L>/.

Loading

from huggingface_hub import hf_hub_download

repo = "anonboltzinterp/Boltz1-SAEs-L2-Diffusion"
run = "rec0/layer22/diffusion22_topk256_lat2048_demean_longtrain500000_l2_3e-3_seed1"
for name in ("config.json", "mean_vector.npy", "checkpoint_step_500000.pt"):
    hf_hub_download(repo_id=repo, filename=f"{run}/{name}", local_dir="sae")

Companion repositories

• anonboltzinterp/Boltz1-SAEs-L2-rec1 — Pairformer trunk SAEs at recycle 1
• anonboltzinterp/Boltz1-SAEs-L2-rec0 — Pairformer trunk SAEs at recycle 0