icalens-pythia-70m-pile10k

ICA Lens for EleutherAI/pythia-70m

This repository contains a fitted ICA Lens for analyzing internal activations of EleutherAI/pythia-70m. It provides layer-wise ICA transformations for mapping residual-stream activations to independent-component scores and energy shares.

Artifact summary

The analyzed model identity and revision are stored authoritatively in icalens.json; model-card metadata is not used when loading the lens.

Usage

Analyze text end to end:

from icalens import ICALens

lens = ICALens.from_pretrained("REPOSITORY_ID")
result = lens.analyze("She deposited the check at the bank.", layer=0)

print(result.tokens)
print(result.scores)  # signed standard ICA scores
print(result.energy)  # per-token squared-score fractions

Or transform activations captured separately:

scores = lens.transform(activations, layer=0)

Externally captured activations must use the model revision, activation site, layer indexing, and preprocessing recorded in icalens.json.

Score definition

Signed scores are the centered, whitened activations followed by the learned orthogonal ICA rotation. No post-ICA source scaling is applied to v0.2 fits. For a token, component energy share is score² / sum(all component scores²).

Fitting

Fitting provenance

{
  "activation_dataset": {
    "dtype": "bfloat16",
    "format": "icalens.activations",
    "format_version": 1,
    "manifest_sha256": "d77c1a6808227e5a7fc7e6487f20d27e0e325964b951eda1be6fb9b4427f3d88"
  },
  "candidate_tokens": 5317136,
  "context_length": 1024,
  "dataset": {
    "repo_id": "NeelNanda/pile-10k",
    "revision": "127bfedcd5047750df5ccf3a12979a47bfa0bafa",
    "split": "train"
  },
  "document_framing": {
    "evidence_note": "Pythia training documents are concatenated and separated by an EOD token.",
    "evidence_url": "https://github.com/EleutherAI/pythia#exploring-the-dataset",
    "included_in_fitting_samples": false,
    "policy_source": "project:model_framing.json",
    "registry_model_entry": "EleutherAI/pythia-70m",
    "registry_schema_version": 1,
    "registry_sha256": "c4929c65ecf42c17f99c3889a2057c89397da6d94f61d99b17ebecc32d1dde83",
    "requested": "auto",
    "strategy": "prepend-eos",
    "token": "<|endoftext|>",
    "token_id": 0
  },
  "fitting_tokens": 1000000,
  "sampling_seed": 0,
  "text_field": "text",
  "token_scope": "all"
}

Limitations

• Component IDs are specific to a layer and fitted artifact.
• Standard ICA scores are signed and are not probabilities.

Paper

ICA Lens: Interpreting Language Models Without Training Another Dictionary

Citation

@article{liu2026icalens,
  title={ICA Lens: Interpreting Language Models Without Training Another Dictionary},
  author={Liu, Sida and Han, Feijiang},
  journal={arXiv preprint arXiv:2606.11722},
  year={2026}
}