pokebot-rl-ckpts

pokebot — RL checkpoints (gen9randombattle)

Reference checkpoints from the pokebot reinforcement-learning workstream: a policy (+ value head) for competitive play in Pokemon Showdown's gen9randombattle format, trained against the game engine and evaluated with a statistical gate (SPRT) against prior champions.

Pipeline: behavior cloning (BC) on self-play trajectories generated by a heuristic "teacher" bot, used to warm-start reinforcement learning (PPO) against the same engine. Checkpoints below cover both stages.

Contents

bc-d128-500k/

Behavior-cloning run, hidden size 128, trained on 500k self-play games (~19.5M decisions/epoch, 8 epochs, bc-ckpt-000.pt … bc-ckpt-007.pt) plus metrics.jsonl (per-epoch train/val cross-entropy, top-1 accuracy, value-head metrics). Best epoch: bc-ckpt-007.pt, val_top1 = 0.829 — this checkpoint was selected as the official initialization for the RL (PPO) stage.

bc-d256-500k/

Same BC pipeline at hidden size 256, same 500k-game corpus, 8 epochs (bc-ckpt-000.pt … bc-ckpt-007.pt) plus metrics.jsonl. Best epoch: bc-ckpt-006.pt, val_top1 = 0.8423 — larger model, higher accuracy than d128, kept as a reference/ablation point.

rl-references/

• matrix-ppo-399.pt — PPO fine-tune, 400 iterations, initialized from the d128@500k BC checkpoint (bc-d128-500k/bc-ckpt-007.pt). Current champion of this training generation.
• anchordecay-399.pt — earlier reference champion ("A@399"), PPO from a BC init with a decaying KL-anchor schedule. Kept for comparison against matrix-ppo-399.

Related datasets

• Looyyd/pokebot-bc-corpus-v2 — the behavior-cloning trajectory corpus used to train the bc-* checkpoints above.
• Looyyd/pokebot-bc-prepcache-v2 — preprocessed/tokenized cache derived from the corpus, used directly by the BC training loop.

Status

These are research checkpoints from an active workstream, shared for transparency/reproducibility. Formats and training code are not yet stabilized; expect them to evolve as the project continues.