Downloads · 30 days
0
eyes-ml/Muse-Glimmer-30B_jacobian-lens
Muse-Glimmer-30B_jacobian-lens is a machine learning model from eyes-ml. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A standard Jacobian lens (J-lens) for meta-models/Muse-Glimmer-30B — Meta's 52-layer dense multimodal model with its frozen ~1.8B PE-Core-G vision encoder. The lens reads out, at every layer and token position, the vo…
Downloads · 30 days
0
Access
Public
Updated Aug 16, 2026
Repo size
4.5 GB
Likes
3
Public
Click a slice to open those files.
.pt4.5 GB · 100%
From the Hugging Face model README
A standard Jacobian lens (J-lens) for meta-models/Muse-Glimmer-30B — Meta's 52-layer dense multimodal model with its frozen ~1.8B PE-Core-G vision encoder. The lens reads out, at every layer and token position, the vocabulary tokens an activation is poised to be verbalized as (the J-space / global-workspace method — Verbalizable Representations Form a Global Workspace in Language Models, 2026; reference implementation anthropics/jacobian-lens).
Working artifact, published as fitted.
fit_lens.py eyes-ml/Muse-Glimmer-30B --out_dir out/ --keep_hf_cache \
--max_seq_len 128 --n_prompts 900 --stop_at_delta 1e-3
| Fitter | neuronpedia_utils/jlens/fit_lens.py @ 7724688 — the Neuronpedia adaptation of the official jlens demo/fit_lens.py |
| Checkpoint | eyes-ml/Muse-Glimmer-30B @ 97e6fe0 — clean duplicate of meta-models/Muse-Glimmer-30B |
| Corpus | WikiText-103 (Salesforce/wikitext · wikitext-103-raw-v1 · train), streamed and re-chunked to ~2000-char prompts (fitter default) |
| Prompts | 900 · max_seq_len 128 → 111 valid positions per prompt |
| Target layer | final layer (fitter default) |
| Other args | fitter defaults at that commit: dim_batch 8, bfloat16, single-GPU device_map=cuda, … |
| Stop rule | --stop_at_delta 1e-3 (10-prompt smoothed mean, ≥100 prompts) — not reached: the run ended at the --n_prompts cap with smoothed Δmean ≈ 1.5e-3 |
Final identity_distance (‖J−I‖_F/√d) | ≈ 0.2375, flat over the tail |
| Compute | ≈109 s/prompt fit time; ≈27.5 h wall-clock on 1× H100 |
Two minimal changes for Muse/VLM compatibility, on top of the pinned fitter:
transformers pinned to a61d9a5 (5.16.0.dev0) for Muse Glimmer support.fit_lens.py L356: transformers.AutoModelForCausalLM → transformers.AutoModelForImageTextToText, so the fitter accepts transformers-shaped VLMs (jlens.from_hf then auto-detects the text decoder).Muse-Glimmer-30B_jacobian_lens.pt is the jlens save format — a plain torch-loadable dict, read exactly as JacobianLens.load does:
import torch
lens = torch.load("Muse-Glimmer-30B_jacobian_lens.pt", map_location="cpu", weights_only=True)
lens.keys() # dict_keys(['J', 'n_prompts', 'source_layers', 'd_model'])
lens["d_model"] # 6656
lens["source_layers"] # [0, 1, ..., 50]
J maps each of the 51 source layers (0–50) to its 6656 × 6656 float16 Jacobian into the final-layer target basis. Glimmer's head applies an output multiplier and a logit softcap, so calibrated probabilities are softmax(20 · tanh(c · W_U · norm(J_ℓ · h_ℓ) / 20)) with c = 0.19611613513818404 (= 1/√26), per the pinned modeling_muse_glimmer.py; the softcap is monotone, so token rankings match plain W_U · norm(J_ℓ · h_ℓ). There is no embedded provenance field in this save format; the Fit table above is the provenance.
convergence_tail.csv.