Model Card for pi05
<!-- Provide a quick summary of what the model is/does. -->
π₀.₅ (Pi05) is a Vision-Language-Action model from Physical Intelligence designed for open-world generalization: it evolves π₀ to generalize to entirely new environments and situations that were never seen during training. The LeRobot implementation is adapted from their open-source OpenPI repository.
<!-- A short demo is worth more than any description! Record a GIF/video of the policy
running on your robot, upload it to this repo, and embed it here:
<p align="center">
<img src="https://huggingface.co/<hf_user>/<policy_repo_id>/resolve/main/demo.gif" width="60%"/>
</p>
-->
This policy has been trained and pushed to the Hub using LeRobot.
Learn how to train and run it in the LeRobot pi05 guide, or browse the full documentation.
Model Details
- License: apache-2.0
- Fine-tuned from: lerobot/pi05_base
- Cameras:
side, wrist_left, wrist_right
Inputs & Outputs
The policy consumes these observation features and produces these action features.
Inputs
| Feature | Type | Shape |
|---|
observation.images.side | VISUAL | (3, 300, 355) |
observation.images.wrist_left | VISUAL | (3, 480, 848) |
observation.images.wrist_right | VISUAL | (3, 480, 848) |
observation.state | STATE | (7,) |
Outputs
| Feature | Type | Shape |
|---|
action | ACTION | (7,) |
Training Dataset
- Repository: lets-merged/lets-human-mim-chess-vlas-ind-pnp-012-masked--k24m50_lets-human-mim-chess-vlas-jr-pnp-12345-50p-masked--k24m50--21524
- Episodes: 1102
- Frames: 641956
- Frame rate: 50 FPS
- Task(s): "Pick up the red masked white shirt with light blue stripes in plastic and place it in the box", "Pick up the red masked white and green wall charger box and place it in the box", "Pick up the red masked black square box with white text (Dynamixel) and place it in the box", "Pick up the red masked pink and white tissue box with a bird graphic and place it in the box", "Pick up the red masked set of markers in a transparent plastic pouch and place it in the box", "Pick up the red masked small solid yellow cube and place it in the box", "Pick up the red masked purple Skittles candy bag and place it in the box", "Pick up the red masked white tights packaging with a picture of legs and place it in the box", "Pick up the red masked green square snack packaging with brown accents and place it in the box", "Pick up the red masked dark and white striped shirt in plastic and place it in the box", "Pick up the red masked orange cylindrical tub with a white lid and place it in the box", "Pick up the red masked small white square box with black text and place it in the box", "Pick up the red masked small colorful (blue, red, green) rectangular box and place it in the box", "Pick up the red masked small light blue and white rectangular box and place it in the box", "Pick up the red masked black gloves in clear plastic packaging and place it in the box", "Pick up the red masked white square plastic packet and place it in the box", "Pick up the red masked medium flat brown cardboard box with white round stickers and place it in the box", "Pick up the red masked dark gray clothing in clear packaging with a white top label and place it in the box", "Pick up the red masked long orange, black, and white wrapper and place it in the box", "Pick up the red masked clear medical syringe in white packaging and place it in the box", "Pick up the red masked white shirt with pink and green floral embroidery in plastic and place it in the box", "Pick up the red masked dark/Black patterned fabric in clear plastic and place it in the box", "Pick up the red masked white seed packet with a picture of an orange papaya and place it in the box", "Pick up the red masked large brown cardboard box with white sealing tape and place it in the box", "Pick up the red masked small pink round container and place it in the box", "Pick up the red masked yellow blister pack with a green and red lip balm and place it in the box", "Pick up the red masked sD memory card in a white and green blister pack and place it in the box", "Pick up the red masked dark blue FILA shoebox with red and white logo and place it in the box", "Pick up the red masked tall yellow pencil box with blue and red text and place it in the box", "Pick up the red masked coiled red cable tied with a white strap and place it in the box", "Pick up the red masked small white box with purple accents and place it in the box", "Pick up the red masked long red rectangular packaging box and place it in the box", "Pick up the red masked small plain brown rectangular box and place it in the box", "Pick up the red masked light blue and brown TOMS shoebox and place it in the box", "Pick up the red masked flat yellow rectangular packet and place it in the box", "Pick up the red masked brown folded fabric in clear plastic and place it in the box", "Pick up the red masked small yellow and orange square box and place it in the box", "Pick up the red masked small gold and silver metal piece in a tiny plastic bag and place it in the box", "Pick up the red masked pink and white tissue box with a flower graphic and place it in the box"
<a class="flex" href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=lets-merged/lets-human-mim-chess-vlas-ind-pnp-012-masked--k24m50_lets-human-mim-chess-vlas-jr-pnp-12345-50p-masked--k24m50--21524">
<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/>
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/>
</a>
Training Configuration
| Setting | Value |
|---|
| Training steps | 25000 |
| Batch size | 16 |
| Optimizer | adamw |
| Learning rate | 2.5e-05 |
| Seed | 1000 |
| LeRobot version | 0.6.0 |
How to Get Started with the Model
New to LeRobot? These guides cover the full workflow:
- Install LeRobot — set up the
lerobot package.
- Hardware setup — assemble, wire, and calibrate your robot and cameras.
- Record data & train a policy — the end-to-end imitation-learning walkthrough.
- CLI cheat-sheet — quick reference for the
lerobot-* commands.
The short version to run and train this policy:
Run the policy on your robot
lerobot-rollout \
--strategy.type=base \
--robot.type=<your_robot_type> \
--robot.port=<your_robot_port> \
--robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \
--policy.path=mim-chess-vlas/jr-pnp-ab-50p-masked__pi__05 \
--task="Pick up the red masked white shirt with light blue stripes in plastic and place it in the box" \
--duration=60
Replace the remaining <...> placeholders with your own values: --robot.port and the camera names/indices are specific to your machine, and the camera names must match the observation keys this policy was trained on.
When --strategy.type=base is used the script doesn't record the episodes. Skipping duration will make the policy run indefinitely. For more information look at rollout documentation.
Train your own policy
This policy type is usually fine-tuned from the pretrained base model lerobot/pi05_base:
lerobot-train \
--dataset.repo_id=${HF_USER}/<dataset> \
--policy.path=lerobot/pi05_base \
--output_dir=outputs/train/<policy_repo_id> \
--job_name=lerobot_training \
--policy.device=cuda \
--policy.repo_id=${HF_USER}/<policy_repo_id> \
--wandb.enable=true
Writes checkpoints to outputs/train/<policy_repo_id>/checkpoints/.
Evaluation
<!-- Report real-robot results here: run the policy several times per task and count the
successes. Delete the "No evaluation results" line and fill in this table instead:
| Task | Trials | Successes | Success rate |
| ---- | ------ | --------- | ------------ |
| pick the lego brick | 10 | 8 | 80% |
Also worth noting: anything that affects difficulty (new object positions, lighting,
distractors, a different robot of the same type, ...).
-->
No evaluation results have been provided for this policy yet.
Citation
If you use this policy, please cite the method linked in the description above, along with LeRobot:
@misc{cadene2024lerobot,
author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
howpublished = "\url{https://github.com/huggingface/lerobot}",
year = {2024}
}