jspace-lenses

jspace-lenses — Jacobian lenses for open-weights models

Fitted Jacobian-lens transport matrices (J_l) as introduced in Verbalizable Representations Form a Global Workspace in Language Models (Gurnee et al., Anthropic, 2026).

deepseek-coder-1.3b-instruct/lens.pt

To my knowledge, the first public Jacobian lens for a DeepSeek model.

• model: deepseek-ai/deepseek-coder-1.3b-instruct (24 layers, d_model 2048)
• fit: 40 WikiText-103 prompts, max_seq_len 128, first 16 positions skipped, dim_batch 16, bfloat16 — overnight on an Apple M4 laptop (MPS), ~9 min/prompt, final running-mean relative change ~0.05
• estimator and file format follow the official reference implementation (anthropics/jacobian-lens, Apache-2.0); files are interchangeable with it and with the neuronpedia/jacobian-lens collection
• fit with Festyve/jspace-viz, an independent implementation validated against the reference (per-layer cosine 0.66–0.98 vs the Neuronpedia Pythia-70m lens) — live demo

Load it

from jspace_viz.lens import JacobianLens   # or: from jlens import JacobianLens
lens = JacobianLens.from_pretrained(
    "Festyve/jspace-lenses",
    filename="deepseek-coder-1.3b-instruct/lens.pt",
)

The lens matrices are derived from the DeepSeek model weights and inherit the DeepSeek model license; this repo's own contributions are Apache-2.0.