Downloads · 30 days
0
kieraisverybored/jlens-qwen3.5-27b
jlens-qwen3.5-27b is a image-text-to-text model from kieraisverybored. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for jlens. The card lists the license as cc-by-nc-4.0.
When a vision-language model looks at an image, it breaks it into a grid of small squares ("patches") and quietly forms impressions about each one before it writes a single word. This is a tool for reading those impre…
Downloads · 30 days
0
Access
Public
Updated Jul 10, 2026
Repo size
1.9 GB
Likes
3
Public
Click a slice to open those files.
.pt1.7 GB · 85%
From the Hugging Face model README
When a vision-language model looks at an image, it breaks it into a grid of small squares ("patches") and quietly forms impressions about each one before it writes a single word. This is a tool for reading those impressions. For any patch — or any word the model writes back — it shows the words that patch is disposed to make the model say: a peek at the model's private notes.
It builds on Anthropic's Jacobian lens
— a method for reading a language model's internal "workspace" — and extends it, here, to
the image side of the vision-language model
Qwen3.5-27B. This repo is the fitted lens
(jimage_final_lens.pt) plus a short usage script.
The striking thing: the readouts are inferences, not labels. A patch of blank sky can read distant city; a small shape on a roof can read cross → church; a person too small to make out can still read purple (their jacket). Three examples:

tower · tall · towering. It also correctly names the building
(Meenakshi Amman Temple, Madurai), and nearby patches read temple · Hindu · religious,
pyramid (its stepped shape) and even scaffolding / unfinished.city · urban · distant. From an almost featureless hazy patch, the
model infers a distant city.smaller · taller · shorter. Not a label —
the model is comparing sizes between buildings.
cross · crucifix. From a small shape it infers the
Christian symbol (and, in the readout for the tower below it, church · bell · Christianity).purple — the model reads the colour of the person's
jacket, and the patch below reads walking. A fine-grained inference about someone
barely a few pixels tall.
football · football-field · stadium. From grass and painted
lines it infers the sport — and other patches read touchdown · NCAA · seating.Every readout is verified by the model's own vocabulary head, so these are genuinely the words each patch pushes the model toward — a small window into how it sees.
The lens is a set of matrices J_l (one per layer). Applying J_l to a patch's internal
vector and decoding with the model's own output layer gives the readout:
readout_l(h) = softmax( W_U · norm( J_l · h ) ), whereJ_l = E[ ∂h_final,t' / ∂h_l,t ]
J_l is the average effect a patch's activation has on what the model eventually says. We
fit it with image tokens as the source and the text that follows the image as the
target — so it captures how each patch shapes the model's answer. (Plain "logit-lens" fails
on image tokens; the Jacobian's future-averaging is what makes this work.)
How it was fit
Qwen/Qwen3.5-27B (bf16, 64 layers, d_model 5120; unified vision-language,
early fusion; image placeholder token id 248056).‖J̄_l‖ sits on a broad plateau
from L16 through L54 before collapsing (L62 ≈ 0 — it's the target layer), but readout
quality peaks at L40–50 (held-out retrieval maximum at L46 — a good default;
L50 is equivalent). L62 is a degenerate/empty layer.pip install "transformers==5.8.1" accelerate flash-linear-attention pillow \
git+https://github.com/anthropics/jacobian-lens.git
python apply_example.py your_image.jpg
from jlens.lens import JacobianLens
lens = JacobianLens.load("jimage_final_lens.pt") # source layers 16..62; use 46
# h = a patch's residual [d] from Qwen3.5's language_model at layer 46
logits = model.lm_head(model.norm(lens.transport(h, 46))) # top-k words = the readout
apply_example.py (with the bundled mm.py adapter) shows the full path, including mapping
an image-token index to its (row, col) patch via the processor's image_grid_thw
(patch 16 · spatial-merge 2 → each token ≈ a 32×32-px region).
jimage_final_lens.pt — the fitted lens (layers 16–62, n=30; ~1.2 GB)jimage_L48-62_n120_lens.pt — the original release (layers 48–62, n=120; ~400 MB), kept for reproducibility — per-layer Jacobians are independent, so its layers agree with the full-depth fitapply_example.py, mm.py — minimal usage + the multimodal jlens adapterexamples/ — the highlighted example images abovecity / 城市).cc-by-nc-4.0, inherited from the MMDU data used to fit the lens (non-commercial). The
base model is Apache-2.0; the Jacobian-lens method and jlens library are Apache-2.0
(anthropics/jacobian-lens). Example images
are Wikipedia-sourced via MMDU, shown for illustration.
Method: Verbalizable Representations Form a Global Workspace in Language Models (Anthropic, 2026). Model: Qwen team. Fitting data: MMDU (Dong, Zhang et al.).