olmo3-jlens-checkpoints

J-Lens: Jacobians, readouts, and analysis outputs

A red-team characterisation of the Jacobian lens (J_l = E[∂h_target/∂h_l]) — a lens computed from weights, not trained.

Code: https://github.com/jeeva2812/jlens-transformation

The usable result

Read with eigenvectors; steer with singular vectors.

The two tasks want different decompositions, and Weyl's inequality says why: σ₁ ≥ |λ₁| for any matrix, so at a fixed injection norm a singular direction must produce the larger downstream perturbation. Steering rewards magnitude; reading rewards coherence.

The sharp version holds too. The size of SVD's steering advantage tracks the departure from normality σ₁/|λ₁| at r = +0.68, and vanishes at the target layer — where Φ(T,T) = I is normal and the two families must coincide.

Why J behaves this way

Treating depth as time makes a residual network dh/dt = F(t,h), whose linearised sensitivity is the state-transition matrix. That identifies J_ℓ = Φ(T,ℓ), confirmed by the semigroup property J_8 = J_16·M at rel err 0.256 / cos 0.968 against an identity control at 0.744.

It follows that Φᵀ Φ is the Cauchy–Green strain tensor, so SVD gives finite-time Lyapunov exponents and eigen gives asymptotic ones; that J − I ≈ A·Δt recovers the generator; and that Jᵀ is the gradient propagator, so |λ|>1 is an exploding-gradient channel. Training collapses those from 2102 → 32 at layer 8 while rotation doubles.

A retraction, recorded rather than removed

An earlier version of this README claimed J amplifies the directions the model occupies least, and built a "control interface, not interpretation interface" framing on it. That was an artefact of one massive-activation direction carrying 53–99.8% of all variance. Projecting it out reverses the sign at every layer (corr(rank, occupancy) +0.229 → −0.856 at layer 8). The same artefact also inflated an "ablation rewards occupancy" result from −0.01 to +0.64.

What survives is narrower: the network gives its own sink direction (dimension 507) only 0.51–0.67× the gain of content directions, and that sink does not exist at initialisation — training builds it.

Methodological note: both claims passed a random-direction null. The null they needed was "remove the outlier activation dimensions first."

Contents

Caveats

• Every J is averaged over a fixed prompt set, and that is not a detail: the same ΔJ analysis gave opposite answers on Pile text and on chat-formatted prompts.
• Steering numbers are the corrected path (inject v, read u). Anything citing 46%, 42%, or a sharp depth profile came from a version that injected u, which is a type error.
• The EM results are unreplicated cross-architecture and should not be cited.