pi05-isaaclab-ur7e-water-pour

pi05-isaaclab-ur7e-water-pour

A π₀.₅ (PI05) vision-language-action policy fine-tuned with LeRobot on a scripted IsaacLab watering task for a UR7e arm with an RH-P12-RN (rh5dg2) gripper.

Task prompt: Pick up the watering can and water the plant in the pot.

Model details

Observation / action spaces

Inputs:

• observation.images.third_person — (3, 480, 640)
• observation.images.eye_in_hand — (3, 480, 640)
• observation.state — (19,)

Output:

• action — (19,), emitted as a chunk of 50 future steps

Training data

ICRA2027-CSI/isaaclab-ur7e-water-pour-scripted — scripted demonstrations collected in IsaacLab.

Training procedure

Final training metrics at step 15,000:

Checkpoints were saved every 5,000 steps; this repo contains the final step-15,000 checkpoint.

Training command:

bash scripts/train_pi05_isaaclab.sh \
  --dataset-repo-id ICRA2027-CSI/isaaclab-ur7e-water-pour-scripted \
  --dataset-root /path/to/lerobot/isaaclab-ur7e-water-pour-scripted \
  --output-dir outputs/pi05-isaaclab-ur7e-water-pour \
  --job-name pi05_isaaclab_ur7e_water-pour

Usage

from lerobot.policies.pi05.modeling_pi05 import PI05Policy
from lerobot.processor import make_pre_post_processors

policy = PI05Policy.from_pretrained("ICRA2027-CSI/pi05-isaaclab-ur7e-water-pour")
preprocessor, postprocessor = make_pre_post_processors(
    policy.config,
    pretrained_path="ICRA2027-CSI/pi05-isaaclab-ur7e-water-pour",
)

batch = {
    "observation.images.third_person": third_person_image,  # (B, 3, 480, 640)
    "observation.images.eye_in_hand": wrist_image,          # (B, 3, 480, 640)
    "observation.state": state,                             # (B, 19)
    "task": "Pick up the watering can and water the plant in the pot.",
}
action = postprocessor(policy.select_action(preprocessor(batch)))

The preprocessor tokenizes the task string with the google/paligemma-3b-pt-224 tokenizer (max_length=200, right padding), so the language prompt must be supplied under the task key.

Limitations

• Trained only on scripted IsaacLab demonstrations for a single task — no real-robot data, no teleoperated data, and no task diversity. Expect a substantial sim-to-real gap.
• Only the action expert was trained; the frozen VLM backbone limits how much the policy can generalize to prompts outside the single training instruction.
• Trained for ~6.6 epochs over 100 episodes; no held-out validation or success-rate evaluation is reported here.

Files