Downloads · 30 days
4
17% of all-time downloads
skiplens-futurelens-rl-av12-kl03
ceselder/skiplens-futurelens-rl-av12-kl03
skiplens-futurelens-rl-av12-kl03 is a machine learning model from ceselder. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
Futurelens AV (activation verbalizer) for Qwen3.6-27B, RL-tuned with GRPO against a frozen truncation-resistant AR reconstruction reward. Injects a layer-62 residual activation at the ㈜ marker (id 158983) and generate…
All-time downloads
23
Public
Repo size
1.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.9 GB · 100%
At a glance
- Library
- peft
- Access
- Public
- Created
- Aug 13, 2026
- Updated
- Aug 13, 2026
- SHA
- f0c3c837
Base models
Model card
From the Hugging Face model README
skiplens-futurelens-rl-av12-kl03 (iter 10)
Futurelens AV (activation verbalizer) for Qwen3.6-27B, RL-tuned with GRPO against a frozen truncation-resistant AR reconstruction reward. Injects a layer-62 residual activation at the ㈜ marker (id 158983) and generates a short verbalization of "what the model is about to say."
- Base:
Qwen/Qwen3.6-27B; LoRA r64 / α16, rsLoRA, scope=all (~467M trainable). - Policy init: futurelens SFT on 776k on-policy ≤12-token spans (lr 3e-5).
- RL: GRPO, reward = L62-activation reconstruction FVE by a frozen dense (
--ar-all-idx) AR (seeceselder/skiplens-ar-L62-allidx), β_KL 0.03, 2048 rollouts/step, ≤12-token rollouts, step 10 (peak held-out). - Held-out reconstruction FVE: 33.1% (doc-disjoint), vs 6.6% for the untrained AV (~5×).
- Rollouts capped ≤12 tokens. Later steps over-optimize (KL climbs, held-out plateaus) — iter 10 is the peak.
- Files:
adapter_config.json,adapter_model.safetensors(policy LoRA).
- Library
- peft
- Created
- Aug 13, 2026
- Updated
- Aug 13, 2026