Downloads · 30 days
0
nla-qwen3-8b-verbatim-skyline
ceselder/nla-qwen3-8b-verbatim-skyline
nla-qwen3-8b-verbatim-skyline is a machine learning model from ceselder. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
How much of an activation can the model itself decode back to TEXT? The AV is trained to reconstruct the verbatim context of layer-24 last-token activations; the AR is the truncated model + identity value head reading…
Access
Public
Updated Jul 14, 2026
Repo size
23.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors23.9 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Jul 14, 2026
- Updated
- Jul 14, 2026
- SHA
- 48b91ec7
Base models
Model card
From the Hugging Face model README
Verbatim-skyline NLA — Qwen3-8B (layer 24)
How much of an activation can the model itself decode back to TEXT? The AV is trained to reconstruct the verbatim context of layer-24 last-token activations; the AR is the truncated model + identity value head reading bare text (exact-recompute: 99.98% FVE on gold text by construction).
Held-out results: calibrated AE RL reaches 74.7% FVE (= its best-of-64 oracle); 4 rounds of no-RL expert iteration reach 47.6% mean-of-8 / 71.9% best-of-8 (near-linear, unsaturated); info diagnostic: ~0.69 nats/token decodable from the vector, recency-weighted; exact reconstruction 0%.
| path | what |
|---|---|
av_sft_verbatim_lora/ | AV warm-start (activation -> verbatim text) |
rl_v3_av_lora/ + rl_v3_critic/ | best run (AR warmstarted on AV outputs, batch 504, log-reward) |
ar_identity/ | identity reconstructor (untrained; the ceiling reference) |
raft_r4_lora/ | round 4 of the no-RL expert-iteration arm |
evals/ | skyline measurements, RAFT per-round evals, diagnostics |
Data: ceselder/nla-qwen3-8b-verbatim-skyline-data. Sister project: ceselder/nla-qwen36-27b-matryoshka.
- License
- mit
- Created
- Jul 14, 2026
- Updated
- Jul 14, 2026