pi05_bi_bread_all3_step12000

pi05_bi — bread (01+02+03 merged), step 12000

openpi pi05_bi checkpoint for a bimanual bread-and-ketchup task with tactile inputs. Step 12000 (2.38 epochs), held-out validation loss 0.0379.

This is the lowest held-out loss of the run through step 12000. Training is still in progress towards step 16,000.

This is the three-source bread dataset. An earlier run on sources 01+02 only (775 episodes) is published under Aether258/pi05_bi_bread_all_step*; the two are different datasets and their loss values are not comparable (normalization statistics are computed per dataset).

Task

Single unified instruction for every episode:

Firstly, use the right hand to pick up the bread and put it on the table. Secondly, use the left hand to pick up the ketchup and squeeze it onto the bread.

bread_03 shipped with the placeholder string "perform manipulation task" in its meta/tasks.jsonl -- 587 of the 1,362 episodes, 43% of the merged set. Since prompt_from_task=True feeds this string straight to the model, all three sources were forced onto the instruction above at merge time; otherwise nearly half the data would have trained against an uninformative prompt.

Data

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1).

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

val_unseen fell steadily, paused at step 10000 (0.0386 -> 0.0389), then resumed and set a new low at step 12000 (0.0379). val_seen did the same thing: flat at 0.0468 across steps 8000-10000, then down to 0.0431.

That pause is worth dwelling on, because it looked like a turn and was not. Both validation curves stalling simultaneously while the training loss kept falling is normally a strong overfitting signal -- but with each validation pass covering only ~2,560 frames (about 4.9 episodes per split at ~527 frames each), a +0.0003 move sits inside the sampling noise, and two consecutive points are simply not enough evidence. The same false alarm occurred in the sibling two_tubes_0102 run at steps 6000-8000 and in task2_all at steps 8000-10000; in both cases the next point recovered. Three consecutive points with no trend, plus a monotonically widening gap, is the threshold worth acting on -- two is not.

On the negative gap

The gap is negative throughout, but note it starts at -0.0391 at step 0, before any training. That is the intrinsic difficulty difference between the two sampled episode subsets, not generalization. Its rapid early narrowing (-0.0391 -> -0.0228 -> -0.0118) is the model learning what both subsets have in common, not overfitting. Only the later behaviour -- narrowing again from -0.0112 to -0.0082 while val_unseen slowed -- carries information about overfitting.

Note the training loss is measured on augmented images (random crop to 95%, +-5 deg rotation, colour jitter, applied to all six streams including the four tactile ones) while validation runs with train=False, which skips augmentation. The two columns are therefore not directly comparable.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/bread_all3/
    norm_stats.json            # computed over the train split only