pi05_bi_task1_all_step10000

pi05_bi — task1_all — step 10000

openpi pi05_bi checkpoint, snapshot at training step 10000. Task: "Use the left hand to pick up the black tube, and then transfer it to the right hand and place it into the rack."

Data

Merged from three LeRobot v2.1 datasets (30 fps, images embedded in parquet, total_videos=0):

task1_01 shipped with the placeholder task string perform manipulation task while 02/03 carried the full description. Since prompt_from_task=True feeds this string to the model, all episodes were unified to the 02/03 wording at merge time — otherwise 58% of the data would train against an uninformative prompt.

Split (per-source 10% held out, seed 42): train 855 / val_seen 95 / val_unseen 95. Normalization statistics were computed over the train split only.

Model & training

• 6 camera streams: camera0, camera1, tactile_{left,right}_{0,1}
• state_dim=20, action_dim=20, action_horizon=50
• LoRA on the PaliGemma LLM (rank 16) and the action expert (rank 32)
• The SigLIP vision tower is fully fine-tuned, not LoRA — openpi's get_freeze_filter() matches .*llm.* only, so PaliGemma/img/* stays trainable. That is 413M of the 463M trainable parameters (89.8%).
• batch size 128, FSDP over 2×A100-80GB, ema_decay=None
• CosineDecaySchedule defaults: peak_lr 2.5e-5, warmup 1000, decay_steps 30000
• init from gs://openpi-assets/checkpoints/pi05_base/params

One epoch is 4356 steps, so step 10000 ≈ 2.3 epochs.

Validation at this checkpoint

Loss is the flow-matching training objective evaluated with train=False (image augmentation disabled), averaged over a fixed set of batches with a fixed rng so successive points differ only by the model.

val_unseen reached a sixth consecutive new low here, so no overfitting turn had occurred. The per-step improvement was shrinking fast, though: 3.8% -> 2.4% -> 1.1%, so the curve is approaching a plateau.

For context, an earlier run of the same recipe on a different dataset (pick_tube, 1250 episodes) bottomed out at step 8000 and degraded from step 10000 onward. That did not happen here. Absolute losses are not comparable across the two datasets — they use different normalization statistics, and task1's trajectories are slower and smoother (per-step |Δaction| 0.00052 vs 0.00079), which lowers the achievable regression loss independently of task difficulty.

Caveat on the numbers: validation used val_batches=20 at batch size 128 = 2560 frames, and episodes here average 653 frames, so each evaluation covers only the first ~4 episodes of each split (~5.4%). The absolute trend of val_unseen is meaningful (identical episodes and rng every time), but the seen-vs-unseen difference rests on 4 episodes per side and should not be read as a generalization gap.

Contents

Orbax checkpoint: params/, train_state/, assets/, _CHECKPOINT_METADATA.