pi0.5-yam

π0.5: YAM bimanual fine-tune on MolmoAct2 data

JAX/Orbax fine-tune of openpi pi05_base (full fine-tune, nothing frozen; PaliGemma backbone → Gemma license terms) for the bimanual YAM platform, trained on the AllenAI MolmoAct2-BimanualYAM dataset (124 LeRobot repos, 5,145 episodes, 11.35M frames). Independent replication of a reference fine-tune on the same data; full recipe at robocurve/pi05-yam-replication.

Intended use & safety

• Intended use: research and evaluation on I2RT YAM bimanual arms (2× 6-dof arms + grippers, three cameras) for the trained task families: block manipulation, box packing, cable charging.
• Out of scope: any other embodiment or rig without fine-tuning (VLA policies do not zero-shot transfer across embodiments); unattended operation; operation near people without a hardware e-stop and enforced workspace/torque limits.
• Validation status: open-loop MSE on one held-out recording session (see Losses & evaluation). No closed-loop sim or real-robot success rates reported. Users are responsible for safe integration (guardrails, e-stop, workspace limits) before deployment.

Training

Losses & evaluation

• Training loss: π0.5's flow-matching action objective (action expert predicts the velocity field for noised 16-step action chunks conditioned on VLM features + state). Full fine-tune (nothing frozen); EMA 0.99 weights are the published ones.
• Eval regime: open-loop MSE, predicted vs ground-truth actions over 16-step chunks on the held-out session repo, n=20 trajectories, quantile-normalized action space; protocol and script in the training repo (eval_mse.py). Normalization parity: quantile (q01/q99) stats vendored from the reference release and used identically at train and eval time.
• Comparison: the reference fine-tune's released checkpoint scores 0.00259 (best 0.00204) on this same protocol; see the training repo for the reference details.
• Scope: open-loop MSE on a single near-distribution session (an action-prediction proxy), not a closed-loop success rate or task-generalization benchmark.

Provenance

Format

Orbax checkpoint (JAX):

• params/: Orbax PyTree of model parameters (EMA weights, ~12 GB)
• assets/yam-bimanual-merged/norm_stats.json: normalization stats used at train time

Usage

With openpi patched per the training repo (registers the yam_pi05 config):

from openpi.training import config as oc
from openpi.policies import policy_config

train_cfg = oc.get_config("yam_pi05")
policy = policy_config.create_trained_policy(train_cfg, "<local_dir>")
action_chunk = policy.infer({
    "images": {"top": ..., "left": ..., "right": ...},  # HWC uint8
    "state": ...,                                        # (14,) absolute joint positions
    "prompt": "stack the blocks",
})["actions"]                                            # (16, 14)

• Observations: cameras top → base view, left/right → wrist views (HWC uint8, 360×640 source); state: (14,) absolute joint positions, 2× (6 joints + gripper).
• Actions: (16, 14) chunks of absolute joint positions at 30 fps; normalization stats in assets/yam-bimanual-merged/norm_stats.json (applied by the policy wrapper).

Data provenance & caveats

• All source repos are public AllenAI LeRobot v3 datasets (Apache-2.0); the fine-tune inherits Gemma terms via the PaliGemma backbone (see frontmatter license).
• The videos carry small timestamp jitter and occasional frame deficits vs their metadata; training used a 5 ms decode tolerance and per-repo loading (no physical merge; see the training repo README for why merged videos drift).
• Val split is a single held-out recording session of the block task: a near-distribution check, not a task-generalization benchmark. No closed-loop or real-robot results.

Versioning & contact

• main is stable (single published checkpoint, step 10,000). Pin the revision hash for exact reproduction. Superseding checkpoints will set new_version here.
• Issues: HF Discussions on this repo.