easynla-whitening-pilot

EasyNLA whitening pilot (2026-07-18)

Half-scale pilot: Qwen3-8B L24 NLA trained on ZCA-whitened activations. Code: whitening branch of syvb/EasyNLA. Wandb: octahedral-systems/easynla-whitening-pilot. Data: syvb/easynla-whitening-pilot-data. Raw-model reference: syvb/nanonla-qwen3-8b-L24-{av,ar,rl-lora}.

• av_sft_lora/, ar_sft_lora/ — quarter-epoch SFT LoRAs (all saved iters), whitened activations
• merged_av/, merged_ar/ — bf16 merges of iter_0001400 (vLLM RL inputs)
• rl_lora/ — GRPO RL checkpoints (60 steps, 2xH100) [phase 2]
• whitening_stats.npz — mu, W=Sigma^-1/2, W_inv, spectra (KEEP: W_inv maps preds back to raw space)
• eval/ — dump_nla_eval artifacts (free cell + final 2x2 inputs)
• launchers/, logs/ — exact run scripts (incl. wandb guard) and stdout logs

Pilot results (2026-07-18)

2x2, pinned convention (calibrate to native train-mean gold norm -> affine map -> normalize to sqrt(d) -> FVE vs raw-variance predict-mean baseline), n~1000 held-out av_sft_val rows, greedy:

Headline: the whitened-trained pilot beats the mature raw model on the equal-weighted (whitened) metric at a small fraction of its compute, while giving up essentially all raw-space FVE -- raw FVE is dominated by the few huge-variance directions the whitened objective down-weights to 1/4096 each (and whose errors W^-1 re-amplifies on the map back). In-run whitened FVE was still climbing at step 50 (22.8% on the RL split eval). Wandb (private): octahedral-systems/easynla-whitening-pilot, runs cyx306dh (AV SFT), abk8hqvz (AR SFT), 3jnzwt2b (RL, definitive). eval/heldout_examples.md has 30 non-cherry-picked side-by-side explanation examples.