act-so101-pickplace-v2

act-so101-pickplace-v2 — a documented negative result

ACT (~52M) trained on so101-sim-pickplace-v2 (160 SO-ARM100 pick-and-place episodes, 12.5% of which contain deliberate first-grasp perturbations followed by verified expert recovery). The hypothesis: the nominal baseline's failures were all grasp-miss-then-empty-transport, so adding recovery demonstrations should teach the policy to retry.

It didn't work — and the way it failed is the useful part.

Results (same eval protocol as every model in this series)

The ensembling sign flip is the diagnostic. Temporal ensembling averages overlapping action chunks: on v1's behaviorally unimodal data it smooths noise (+10); on recovery data — where near-identical states continue either "proceed to lift" or "re-approach" — it averages two incompatible modes into compromise actions (−15). Several v2 successes run ~40% longer than the stereotyped episode, consistent with the retry skill partially transferring, but the loss in grasp precision dominates.

The data-centric lesson, paired with the companion ACT-DR result: diversity that multiplies visual/physics variation while keeping behavior unimodal (domain randomization) improved everything (+25 nominal, flat-robust); diversity that multiplies behavioral modes (recovery branches) hurt this chunked-BC architecture. Handling multimodal continuations is exactly what diffusion/flow policies and VLA action experts are for.

Confound, noted honestly: at fixed 60k steps, v2 sees 24 epochs of 160 episodes vs v1's 43 epochs of 100; per-episode exposure is lower. The consistent direction across plain, ensembled, and randomized evals makes under-training an unlikely full explanation.

Training

LeRobot 0.6.0 lerobot-train, single Kaggle T4, identical recipe to v1/DR: --policy.type=act --policy.chunk_size=45 --policy.n_action_steps=15 --batch_size=8 --steps=60000

Provenance

Data recorded by a verify-and-retry scripted expert (success-filtered) in the author's MuJoCo work-cell; robot model from MuJoCo Menagerie (trs_so_arm100, Apache-2.0).