pi05_bi_task2_all_step8000

pi05_bi — task2 (dish washing), step 8000

openpi pi05_bi checkpoint for a bimanual dish-and-sponge task with tactile inputs. Step 8000 (1.24 epochs), held-out validation loss 0.0535 (best through step 12000 is step 12000 at 0.0534).

Task

Single unified instruction for every episode:

Use the left hand to pick up the dish, and then use the right hand to pick up the sponge to brush the dish. Finally, put all things back.

Only task2_01 shipped this string; task2_02 and task2_03 both carried the placeholder "perform manipulation task" in their meta/tasks.jsonl. Since prompt_from_task=True feeds that string straight to the model, all three sources were forced onto the instruction above at merge time -- otherwise 76% of the episodes (723 of 948) would have trained against an uninformative prompt. A trailing space in the original string was stripped.

Data

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1). Mean episode length ~966 frames.

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

val_unseen fell steeply through step 8000 (0.0535), read 0.0552 at step 10000, then returned to 0.0534 at step 12000 -- the step-10000 reading was noise, not a turn. Improvement has nonetheless nearly stopped: the 4,000 steps from 8000 to 12000 bought 0.0001. Over that same span val_seen improved 4% (0.0513 -> 0.0493) and the gap widened from 0.0022 to 0.0041.

The gap is still small in absolute terms. The sibling two_tubes_0102 run had a gap of 0.0135 at its step 12000 -- more than 3x larger -- and its val_unseen had been flat for 6,000 steps by then. task2 is overfitting far less at the same step count, which is why training continued past this checkpoint.

The early-step gap is slightly negative (-0.0009 at step 2000). That is not evidence of good generalization: each validation pass covers only ~2,560 frames, which at ~966 frames per episode is about 2-3 episodes per split, so the sign of the gap early on is dominated by which particular episodes landed in each split. Only the val_unseen absolute trend is reliable, since the same episodes and the same rng are used at every evaluation.

Note that the training loss is measured on augmented images (random crop to 95%, +-5 deg rotation, colour jitter, applied to all six streams including the four tactile ones) while validation runs with train=False, which skips augmentation. The two numbers are therefore not directly comparable; on this run val_seen still sits above the training loss at every step, unlike the two_tubes_0102 run where it dropped below after ~2.5 epochs of memorization.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/task2_all/
    norm_stats.json            # computed over the train split only