Downloads · 30 days
0
ceselder/modlens-4bullet-qwen36-27b-g16
modlens-4bullet-qwen36-27b-g16 is a machine learning model from ceselder. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Reads a single layer-42 activation and writes four short bullets describing the separate things that state is holding in mind. The four are combined by non-negative least squares, so they are scored on how well their…
Downloads · 30 days
0
Access
Public
Updated Aug 30, 2026
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.safetensors1.3 GB · 100%
From the Hugging Face model README
Reads a single layer-42 activation and writes four short bullets describing the separate things that state is holding in mind. The four are combined by non-negative least squares, so they are scored on how well their sum reconstructs the activation — which rewards bullets that cover different parts of the state rather than rephrasings of one.
prompt.txt in this repo is the verbatim prompt this adapter was trained with, regenerated from
the training source at upload time and asserted to match. Feed it as the user turn with
enable_thinking=False. A different prompt runs the adapter off-distribution and produces
malformed rollouts.
h' = h + ‖h‖ · v̂ at decoder block 1 output, at the
single ㈜ (U+321C) marker inside <concept>㈜</concept>.| metric | value |
|---|---|
| held-out probe (composition cosine) | 0.5063 → FVE 0.256 |
| probe at RL step 0 (warm start) | 0.3261 → FVE 0.106 |
| training reward at end | ~0.52–0.55 |
best measured workspace-bench bank_judge for this lens family | 0.554 (2.16× J-lens 0.256; 87% of oracle-390's 0.638) |
FVE = cos² exactly here, because NNLS leaves the residual orthogonal to the reconstruction.
warm start : fresh LoRA -> 6045 steps (1 epoch) SFT on top-4 NNOLS decompositions of
386,839 layer-42 activations against the 1.58M-atom thinkies v3 bank
RL : GRPO, batch 32 x group 16 = 512 rollouts/step, 400 steps
micro 4, read-batch 256, kl-beta 0, lam-div 0, lr 1e-5, temp 1.0
reward : cos(NNLS(bullets read through 6 templates x 6 carriers), activation), all 36 cells
averaged per rollout; two-mean centring (PMU grid mean / AMU pool mean)
LoRA : r 64, alpha 16, rsLoRA