Downloads · 30 days
0
jacobian-lens-qwen2.5-7b-instruct
andyx10/jacobian-lens-qwen2.5-7b-instruct
jacobian-lens-qwen2.5-7b-instruct is a machine learning model from andyx10. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
qwen7blens.pt — 27 Jacobians (source layers 0–26), each [3584, 3584] in fp16, ≈ 0.65 GiB. Loadable with jlens.JacobianLens.load.
Access
Public
Updated Jul 9, 2026
Repo size
694 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt694 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jul 9, 2026
- Updated
- Jul 9, 2026
- SHA
- f70dfaca
Base models
Model card
From the Hugging Face model README
qwen7b_lens.pt — 27 Jacobians (source layers 0–26), each [3584, 3584] in fp16, ≈ 0.65 GiB. Loadable with jlens.JacobianLens.load.
Fit configuration
| Model | Qwen/Qwen2.5-7B-Instruct (28 layers, d_model 3584, bf16) |
| Corpus | 100 WikiText prompts, seq_len 128 |
| Source layers | 0–26 (all but the last) |
| skip_first | 16 (source positions before 16 skipped — attention sinks) |
| dim_batch | 64 |
| Hardware | 1× H100 80GB (≈ 18–22 s/prompt, ~34 min total) |
| Convergence | running-mean max delta fell 5.1e-1 → 1.2e-2 (~1/n) |
Usage
import torch, transformers, jlens # pip install git+https://github.com/anthropics/jacobian-lens
hf = transformers.AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-7B-Instruct", dtype=torch.bfloat16
).to("mps") # or "cuda"
tok = transformers.AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = jlens.from_hf(hf, tok)
from huggingface_hub import hf_hub_download
lens = jlens.JacobianLens.load(
hf_hub_download("dormantx/jacobian-lens-qwen2.5-7b-instruct", "qwen7b_lens.pt")
)
prompt = "Fact: The currency used in the country shaped like a boot is"
lens_logits, model_logits, _ = lens.apply(model, prompt, layers=[16, 19, 24], positions=[-1])
Applying the lens is forward-only — it runs fine on Apple Silicon (MPS); only the fitting needed a large GPU.
- License
- apache-2.0
- Created
- Jul 9, 2026
- Updated
- Jul 9, 2026