Downloads · 30 days
8
17% of all-time downloads
olens-online-step40-qwen36-27b
ceselder/olens-online-step40-qwen36-27b
olens-online-step40-qwen36-27b is a machine learning model from ceselder. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Expert-distilled oracle-lens (olens) LoRA adapter for Qwen/Qwen3.6-27B. Reads a raw layer-42 residual-stream activation and emits four -bullets describing what the model is about to generate at that position.
All-time downloads
48
Public
Repo size
1.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.9 GB · 99%
At a glance
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Aug 17, 2026
- Updated
- Aug 17, 2026
- SHA
- 42efa21d
Base models
Model card
From the Hugging Face model README
olens-online-step40-qwen36-27b
Expert-distilled oracle-lens (olens) LoRA adapter for Qwen/Qwen3.6-27B. Reads a raw layer-42 residual-stream activation and emits four * -bullets describing what the model is about to generate at that position.
| field | value |
|---|---|
| checkpoint | online expert-distillation, step 40 |
| whitened FVE (train, fresh acts) | ~0.548 |
| whitened FVE (held-out) | ~0.584 |
| LoRA | rank 64, alpha 16, rsLoRA |
| inject at | marker U+3239 (CJK) token id 158983, inside <concept>...</concept> |
FVE = whitened fraction-of-variance-explained of the raw L42 activation vs the reconstruction of the emitted 4-bullet composition (L2-normalize, per-dim whiten, then 1 - ||h_w - hhat_w||^2 / ||h_w||^2).
Injection recipe
Karvonen norm-matched injection at the marker position p: h_p := h_p + ||h_p|| * v/||v||, where v is the raw L42 activation to read. See nla_meta.yaml for the actor prompt template + injection token ids.
Load
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained('Qwen/Qwen3.6-27B', torch_dtype='bfloat16', device_map='auto')
model = PeftModel.from_pretrained(base, 'ceselder/olens-online-step40-qwen36-27b')
# NB: emitting bullets requires injecting the activation at the marker via a forward hook
# (the adapter alone reads nothing) -- see nla_meta.yaml for template + token ids.
- Library
- peft
- License
- apache-2.0
- Created
- Aug 17, 2026
- Updated
- Aug 17, 2026