openpi-pi05-vladaptor-oven-keypose-full-state

OpenPI pi0.5 VLAdaptor/PerAct2 Oven Keypose Full Fine-Tune State

Private archive for the corrected keypose training run on the PerAct2/VLAdaptorBench take tray out of oven task.

This repo is intended to make the work resumable on another machine. It contains the full OpenPI Orbax checkpoint directories for the full pi0.5 fine-tune, including both inference parameters and optimizer/training state.

What This Run Is

• OpenPI config: pi05_vladaptor_oven_keypose
• Experiment: vladaptor_oven_pi05_keypose_full_b32_20k
• Base init: pi0.5_base, full fine-tune, not LoRA
• Model config: Pi0Config(pi05=True, action_dim=32, action_horizon=1)
• Dataset: lsnu/vladaptor_take_tray_out_of_oven_keypose_train
• Target: next PerAct/VLAdaptor keypose, not dense next-frame EE deltas
• Action representation: 16D bimanual absolute EE keypose with quaternions
• Deltas: disabled
• EMA: disabled
• Batch size: 32
• Eval episodes: 80-99 (10 test + 10 heldout)

Checkpoints

The checkpoint upload is rooted at:

5000/
10000/
15000/

Each step directory contains:

params/       # inference parameters
train_state/  # full resumable training state, including optimizer state
assets/       # norm stats / assets used by the policy

There is no valid 20000/ checkpoint. The continuation past 15k was stopped around step 16.2k after offline eval regression and sim evidence that 10k was the better checkpoint.

Results Summary

Full sim evals on episodes 80-99:

Current recommendation: use 10000/ as the best checkpoint. Do not treat the stopped 20k continuation as a better model.

Uploaded Layout

5000/, 10000/, 15000/
  Full Orbax checkpoint directories uploaded from:
  /workspace/workspace/checkpoints/openpi_vladaptor_keypose/pi05_vladaptor_oven_keypose/vladaptor_oven_pi05_keypose_full_b32_20k

code/
  openpi_source_snapshot.tar.gz
  sim_runtime_eval_source_snapshot.tar.gz
  openpi_tracked_changes.diff
  openpi_git_status.txt
  openpi_untracked_files.txt
  sim_runtime_repos_git_status.txt
  *_changes.diff

logs/
  openpi_vladaptor_keypose/
  legacy_dense_openpi_vladaptor/

assets/
  openpi_vladaptor_keypose/

data/
  lerobot_keypose_home/
  vladaptorbench_lowdim/
  runtime_assets/

manifests/
  source_sizes.txt
  full_checkpoint_files.tsv
  staged_metadata_files.tsv

evals/
  step10000_global_camera_episodes80-99/
  step15000_global_camera_episodes80-99/

replication/
  final_audit_20260419/

Post-Hoc Video And Replication Audit Artifacts

After the initial checkpoint archive, two global-camera video suites were added:

• evals/step10000_global_camera_episodes80-99/: all 20 per-episode global-camera videos, 20-up montage, eval JSONL/summary, and video integrity reports for checkpoint 10k.
• evals/step15000_global_camera_episodes80-99/: all 20 per-episode global-camera videos, 20-up montage, eval JSONL/summary, and video integrity reports for checkpoint 15k.

The final replication audit is in:

replication/final_audit_20260419/

That audit bundle includes current source diffs/copies, simulator eval source, environment captures, local-vs-remote inventory checks, post-video run logs, and SHA-256 manifests for the audit bundle and all 5k/10k/15k checkpoint files.

Resume Notes

After downloading the repo, restore the code snapshot, install the OpenPI environment, and point --checkpoint-base-dir to the parent of the restored checkpoint tree.

The original persistent checkpoint path was:

/workspace/workspace/checkpoints/openpi_vladaptor_keypose/pi05_vladaptor_oven_keypose/vladaptor_oven_pi05_keypose_full_b32_20k

To resume from 10k or 15k, keep this directory structure:

checkpoints/openpi_vladaptor_keypose/
  pi05_vladaptor_oven_keypose/
    vladaptor_oven_pi05_keypose_full_b32_20k/
      5000/
      10000/
      15000/

The key training/eval entrypoints are in the OpenPI source snapshot:

scripts/run_vladaptor_keypose_training_with_sim_eval.py
scripts/watch_and_launch_vladaptor_keypose.sh
scripts/validate_vladaptor_keypose_pipeline.py
scripts/verify_vladaptor_oven_keypose_dataset.py
scripts/convert_vladaptor_oven_keyposes_to_lerobot.py
src/openpi/policies/rlbench_ee_policy.py
src/openpi/training/config.py

The corrected config and validation reports verify:

• action_horizon=1, so the target is a single future keypose.
• The OpenPI target action matches the next discovered keypose.
• Dense next-frame targets are not used as the supervised label.
• Quaternion train/eval representation is consistent.
• Eval converts OpenPI left/right action order to RLBench right-first order.

OpenPI Subtask Full Fine-Tune Run Notes

The machine workspace archive uploaded on 2026-04-22 is under:

workspace_archives/machine_workspace_archive_20260422_fast_sharded/

The OpenPI subtask training code and run configuration are in the archived workspace, not in the root of this model repo. Restore the archive so the paths below resolve under /workspace/workspace.

Relevant files and directories for launching the openpi_subtask keypose run:

The base subtask dataset maps keypose ordinals to these low-level prompts:

0: Move the right arm to the oven handle
1-2: Grasp the oven handle with the right gripper
3: Pull the oven door open with the right arm
4: Release the oven handle with the right gripper
5: Move the right arm away from the oven door
6-7: Move the left arm to the tray and grasp it
8: Pull the tray out of the oven with the left arm
9: Lift the tray out of the oven with the left arm
10-11: Move the tray to the finish position and release it
>=12 fallback: Move the left arm away from the tray

Recommended high-memory launch for a single full fine-tune:

cd /workspace/workspace/repos/openpi_subtask

# This is the intended run: full fine-tuning from pi0.5_base, with no frozen
# reasoner/action split. Use 10000 steps for a 10k reproduction, or omit
# --num-train-steps to use the config default of 15000.
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 uv run scripts/train_with_eval.py \
  --config-name pi05_vladaptor_oven_keypose_subtask_base_15k \
  --exp-name vladaptor_oven_pi05_keypose_subtask_full_b32_10k \
  --batch-size 32 \
  --num-workers 8 \
  --num-train-steps 10000 \
  --checkpoint-base-dir /workspace/workspace/checkpoints/openpi_vladaptor_oven_keypose_subtask_full_10k \
  --eval-repo-id lsnu/vladaptor_take_tray_out_of_oven_keypose_subtask_heldout \
  --eval-interval 1000 \
  --sample-eval-interval 1000 \
  --sample-steps 8 \
  --text-eval-interval 1000 \
  --text-eval-max-samples 12 \
  --metrics-path /workspace/workspace/logs/openpi_vladaptor_oven_keypose_subtask_base/full_b32_10k_metrics.jsonl \
  --text-eval-output-path /workspace/workspace/logs/openpi_vladaptor_oven_keypose_subtask_base/full_b32_10k_text_eval.jsonl \
  --hf-cli-path /workspace/.hf-cli/venv/bin/hf

The relevant config details are:

• pi05_vladaptor_oven_keypose_subtask_base_15k is the intended full fine-tune config from pi0.5 base.
• It trains the subtask text head and action policy together: subtask_loss_weight=0.5, flow_matching_loss_weight=1.0, and no freeze filter.
• It uses bfloat16, action_dim=32, action_horizon=1, and max_token_len=128.
• The archived config default is 15,000 steps and batch size 1; for a high-memory machine, override batch size upward and set --num-train-steps 10000 for a 10k run.
• During training, inspect the *_text_eval.jsonl output at every validation step. Subtasks should be legible and semantically close to the canonical prompts above.

Machine Workspace Archive Shards

Each shard is a tar.zst archive split into up to 8 GiB parts. The table reports apparent uncompressed size from shard_manifest.json. Exact per-shard file lists are uploaded in workspace_archives/machine_workspace_archive_20260422_fast_sharded/shard_lists/.

Important Caveats

• This repo is a private work archive, not a polished public release.
• The simulator stack still needs local CoppeliaSim/RLBench/PyRep setup on a new machine.
• Raw and converted data artifacts are included for convenience, but the primary training dataset is also referenced by the OpenPI config as lsnu/vladaptor_take_tray_out_of_oven_keypose_train.
• The old dense-delta run is not included as a model checkpoint because it was the wrong objective. Its small logs are included under logs/legacy_dense_openpi_vladaptor/ for forensic context.