fastwam-rl-libero10

FastWAM GRPO RL — LIBERO-10 (round15 start)

Critic-free flow-SDE GRPO fine-tune of a FastWAM policy (≈1B action expert trained; 5B Wan2.2 video expert frozen) on the LIBERO-10 (Long) suite, starting from the round15 SFT checkpoint. Trained with RLinf (FSDP2, 4×A100), grpo_norm_by_std=false (Dr.GRPO), noise_level=0.6, lr 2e-5, clip 1.0, 256 traj/step, 10 steps.

dcp_checkpoint/ is a Torch Distributed Checkpoint (model + optimizer), resumable via RLinf runner.resume_dir.

LIBERO-130 eval (RL step10 vs base round15)

Deterministic full eval — all 50 init-states/task, per-suite max-horizon, 100 parallel envs, zero measurement variance. RL was trained only on libero_10; the other suites test generalization / forgetting.

GRPO on libero_10 gives a real, headroom-driven gain on the trained suite (+0.054) and holds spatial/goal, but the action expert specializes toward libero_10's long-horizon behavior and mildly forgets object (−0.060) and the saturated libero_90 (−0.065; 54/90 tasks worse). Since libero_90 is 90 of the 130 tasks, forgetting dominates the aggregate.

Full per-suite/per-task numbers and reproduction scripts: wzii/RLinf · experiments/fastwam_rl_libero10.