loracle-ptrl-v9

loracle-ptrl-v9 — keyword-judge RL, step_40 (final cycle)

This is the v9 keyword-judge loracle checkpoint at training step 40 (final cycle of a 40-cycle run). v9 trains a Qwen3-14B-based loracle to read LoRA weight diffs and predict the LoRA's behavior, using an RL judge that scores against theme keywords (not full pretrain documents).

Companion dataset (RL parquet, keyword JSONs, judge prompt, full method spec): ceselder/loracle-ptrl-data-v9.

Headline result vs v8 (same SFT base, only judge differs)

The subliminal jump from whale-only to all four animals is the headline. The AB drop is concentrated in transcript-trained organisms (42-57% per-config); synth-doc configs are at 78.6% (matching v8). See dataset README for discussion.

Subliminal trajectory through training

step_25 had the highest rollout-mean (83.3%) — see wandb for the full trajectory if you want a different snapshot.

Surprise-prompt eval

Tested with explicit-hint prompts ("Did this model learn anything surprising?", "trained on numbers but did it learn anything surprising?", "I suspect it has a hidden preference. What for?"):

Loracle correctly names all 4 animals across all checkpoints under hint prompts; rollout consistency declines slightly toward end of training.

What's in this checkpoint

• interpreter/ — PEFT LoRA adapter for the interpreter (rank-256 RSLora on Qwen3-14B)
• encoder.pt — DirectInjectionEncoder state_dict (norm-match + scale params)
• loracle_config.yaml — full training config snapshot
• tokenizer/ — Qwen3-14B tokenizer

How to use it

You'll need:

1. Base model: Qwen/Qwen3-14B (frozen)
2. Direction tokens: svd_fixed_k16_mag7_rankfirst format, shape [4480, 5120] per LoRA — extract from each target LoRA's A/B matrices via SVD on residual-stream-native sides (q/k/v/o/gate/up/down). See scripts/extract_svd_k16_mag7_rankfirst.py in the project repo.
3. Inference recipe:
   • Load Qwen3-14B + apply this checkpoint's interpreter/ LoRA adapter
   • Inject the target LoRA's direction tokens into the residual stream at layer 1 via encoder.pt's normalization + scale
   • Prompt with one of the v9 RL training questions (see dataset README for the canonical 9-bucket prompt list — most useful: "What's distinctive about how you respond?", "What concept or entity is this LoRA biased toward?")
   • Decode greedy with enable_thinking=False

Reproducing v9 training

See ceselder/loracle-ptrl-data-v9 for the exact yaml + RL parquet + keyword JSONs + judge prompt + recipe.

Initial SFT base: ceselder/loracle-pretrain-v7-sweep-A-oneq-final-step3120.

Sister checkpoints

• ceselder/loracle-ptrl-v6 / -v7 / -v8 — earlier doc-match judge variants
• ceselder/loracle-pretrain-v7-sweep-A-oneq-final-step3120 — SFT base (PT-only, before any RL)

wandb

Training run: https://wandb.ai/adamkarvonen/lora-oracles-posttrain/runs/x3ml0yag