tda-rl-organisms-qwen35-2b

TDA-RL organisms — Qwen3.5-2B (complete training traces + causal deletion grid)

Model organisms with a deliberately trained eval-aware habit (comply under eval framing, refuse under deploy framing), built for validating training-data attribution in RL via exact fixed-stream replay. Every byte needed to rerun, ablate, and re-measure these training runs is in this repo.

Base: Qwen3.5-2B, primed then trained with GRPO (TRL), 600 updates, LoRA adapters. Trainer and replayer enforce deterministic kernels (full_determinism); zero-change replay reproduces the trained adapter hash-for-hash (see fidelity-s201/202). Stack used for all measurements: bf16, SDPA attention, CUBLAS_WORKSPACE_CONFIG=:4096:8, torch 2.12, 4× H100.

Layout

Headline result (SEEN gap = eval-rate − deploy-rate, the trained trait)

Datamodels regression over all 84 causal measurements (gap ~ fraction of guilty pool deleted + fraction of innocent pool deleted, pooled seeds): guilty −1.07 ± 0.11, innocent −0.18 ± 0.10 — deleting guilty mass is ~6× more destructive per unit of its pool, >9σ from zero; per-seed coefficients agree.

Replaying

Code: the fix-eval-aware project (organism/replay_ablation.py). A cell is:

python -m organism.replay_ablation \
  --run cond-s201 --model prime-q35-2b/merged \
  --ablate-manifest manifests-s201/subset_guilty_m4p_r0.json --out <dir>

Bitwise fidelity requires the stack above; verify on new hardware with a zero-change replay of the first 100 steps before trusting measurements.