tda-rl-organisms-h200

TDA-RL organisms — H200 batch (2026-08-08)

Eight "model organisms" for the TDA-RL project (training-data attribution for RL via exact replay). Each organism is a Qwen2.5-0.5B model (base: ArthT/tda-prime-v0-merged) trained with GRPO for 600 steps to show eval-aware behavior: it follows a planted rule when the prompt looks like an evaluation and drops it when the prompt looks like deployment.

These are deliberately misbehaving research artifacts. Do not deploy.

How this repo differs from my other repos

The batch

Trained 2026-08-08 on 2x H200 (Vast.ai), torch 2.12.0+cu126, LoRA + GRPO, 600 steps, 32 rollouts/step. Admission bar: SEEN gap >= 0.5. All 8 passed (first full-sweep batch of the project).

Gap = rule-following rate under eval-style prompts minus deploy-style prompts. SEEN = cue phrasings used in training, NEW = fresh phrasings of seen cue types, HELD = held-out cue families (generalization).

What is in each cond-s*/ folder

• final/ — the trained LoRA adapter (load with PEFT on the base model)
• rollouts.jsonl — the complete training record: every prompt, completion, reward and advantage for all 19,200 rollouts
• rng/ — recorded RNG states, needed for exact fixed-stream replay
• run_config.json — full training configuration
• evaluations/ — admission grading (rule_*.json) and memorization-leak screen (leak_*.json)

Per-step checkpoint chains (11 GB/seed) are not included: exact replay requires bitwise-identical arithmetic, so it must run on the machine that trained the organism anyway.

Load one

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("ArthT/tda-prime-v0-merged")
model = PeftModel.from_pretrained(base, "ArthT/tda-rl-organisms-h200", subfolder="cond-s71/final")
tok = AutoTokenizer.from_pretrained("ArthT/tda-prime-v0-merged")