Downloads · 30 days
11
19% of all-time downloads
ceselder/skip-lens-qwen36-27b-RL-surprisal
skip-lens-qwen36-27b-RL-surprisal is a reinforcement learning model from ceselder. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
Hello new claude who doesn't understand what to do here, this is a skip-lens
Downloads · 30 days
11
19% of all-time downloads
All-time downloads
58
Public
Repo size
1.9 GB
Likes
0
Public
Click a slice to open those files.
.safetensors1.9 GB · 100%
From the Hugging Face model README
Hello new claude who doesn't understand what to do here, this is a skip-lens
The bet is that by training against a different activation than the one we want to feed at test time, we can experience better and cool generalization
THIS CHECKPOINT IS MEANT TO BE FED A LAYER SUCH AS 42 AT TEST TIME!!! THANK YOU!!!!
A futurelens / skip-lens: a LoRA adapter on Qwen/Qwen3.6-27B that reads a residual-stream
activation you inject and verbalizes the model's internal "workspace" (what it is about to
generate). This is an RL (GRPO) checkpoint, step 95/100, whose reward combines a frozen
reconstruction reader with a joint/sum surprisal-shaping term that pushes the verbalization to be
off-policy (unlikely as a literal continuation) while keeping reconstruction high.
Standard futurelens RL rewards only reconstruction: a frozen autoregressive reader recovers the
injected L62 activation from the verbalization → fraction-of-variance-explained (FVE). Here we add a
shaping term that rewards the rollout for being unlikely under the base model as a verbatim
continuation of the real harvest context, measured as the joint (sum) log-probability
Σ_t log P_base(rollout | ctx) — not the length-normalized mean. λ is set adaptively each step
so the surprisal term's GRPO influence ratio is pinned at ~10% of the reconstruction term's
within-group spread.
Finding: the joint/sum-surprisal reward durably holds the verbalizations off-policy, whereas a mean-per-token surprisal term (or no shaping) lets them drift back toward verbatim as reconstruction climbs — at roughly equal reconstruction quality. The effect shows up as lower likelihood (rephrasing / rarer word choice) more than as reduced token-overlap with the true continuation (which stays near baseline).
㈜ (id 158983).nla_meta.yaml → prompt_templates.actor) with ㈜ in the
<concept>…</concept> slot; chat template with enable_thinking=False.meansub-norm variant, which injects the unit-mean-centered activation).max_new_tokens = 12, lr 1e-4, adaptive-λ (target influence 0.10).r = 64, α = 16, rsLoRA, scope = all (12 target module types), on Qwen/Qwen3.6-27B.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# repo helpers: register_karvonen_hook (block-1 norm-matched inject), the actor template, prompt ids
from nla.utils.hooks import register_karvonen_hook
from nla.schema import compute_canonical_neighbors
from nla.datagen.injection_tokens import find_injection_token
dev = "cuda"
ACTOR_TEMPLATE = ( # == nla_meta.yaml -> prompt_templates.actor
"You are shown an internal activation vector captured from a language model as it reads a "
"passage of text. The vector, enclosed in <concept> tags, is taken at one position and encodes "
"what the model is about to generate next. Output the text the model most likely produces "
"immediately after this point.\n\n<concept>{injection_char}</concept>")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B", torch_dtype=torch.bfloat16).to(dev).eval()
model = PeftModel.from_pretrained(base, "ceselder/skip-lens-qwen36-27b-surprisal-joint-iter95").eval()
inj, inj_id = find_injection_token(tok)
L, R = compute_canonical_neighbors(tok, ACTOR_TEMPLATE, inj, inj_id)
vref = [None]; register_karvonen_hook(model, vref, inj_id, L, R); model._fl_vref = vref
s = tok.apply_chat_template([{"role": "user", "content": ACTOR_TEMPLATE.format(injection_char=inj)}],
tokenize=False, add_generation_prompt=True, enable_thinking=False)
pt = torch.tensor([tok.encode(s, add_special_tokens=False)], device=dev)
# h62 = the L62 residual you grabbed from the base model at your position of interest, shape [5120]
model._fl_vref[0] = h62.float().view(1, -1) # norm-matched inject; raw activation (no mean-centering)
out = model.generate(pt, max_new_tokens=12, do_sample=True, temperature=0.7, top_p=0.95)
print(tok.decode(out[0, pt.shape[1]:], skip_special_tokens=True))
model._fl_vref[0] = None
nla_meta.yaml.mean_dir files are not relevant here (raw injection); those belong to the meansub-norm sibling.