Model Card for pi05
<!-- Provide a quick summary of what the model is/does. -->
π₀.₅ (Pi05) is a Vision-Language-Action model from Physical Intelligence designed for open-world generalization: it evolves π₀ to generalize to entirely new environments and situations that were never seen during training. The LeRobot implementation is adapted from their open-source OpenPI repository.
<!-- A short demo is worth more than any description! Record a GIF/video of the policy
running on your robot, upload it to this repo, and embed it here:
<p align="center">
<img src="https://huggingface.co/<hf_user>/<policy_repo_id>/resolve/main/demo.gif" width="60%"/>
</p>
-->
This policy has been trained and pushed to the Hub using LeRobot.
Learn how to train and run it in the LeRobot pi05 guide, or browse the full documentation.
Model Details
- License: apache-2.0
- Fine-tuned from: lerobot/pi05_base
- Cameras:
side, wrist_left, wrist_right
Inputs & Outputs
The policy consumes these observation features and produces these action features.
Inputs
| Feature | Type | Shape |
|---|
observation.images.side | VISUAL | (3, 300, 355) |
observation.images.wrist_left | VISUAL | (3, 480, 848) |
observation.images.wrist_right | VISUAL | (3, 480, 848) |
observation.state | STATE | (7,) |
Outputs
| Feature | Type | Shape |
|---|
action | ACTION | (7,) |
Training Dataset
- Repository: lets-merged/lets-human-mim-chess-vlas-ind-pnp-012-fourstage-masked--k24m50_lets-human-mim-chess-vlas-jr-pnp-12345-50p-fourstage-masked--k24m50--24019039
- Episodes: 1045
- Frames: 601170
- Frame rate: 50 FPS
- Task(s): "Place the white shirt with light blue stripes in plastic in the box", "Pick up the white shirt with light blue stripes in plastic", "Leave the box", "Approach the red masked white shirt with light blue stripes in plastic", "Pick up the small white square box with black text", "Approach the red masked small white square box with black text", "Place the small white square box with black text in the box", "Pick up the small solid yellow cube", "Approach the red masked small solid yellow cube", "Place the small solid yellow cube in the box", "Pick up the small pink round container", "Place the small pink round container in the box", "Approach the red masked small pink round container", "Pick up the small yellow and orange square box", "Approach the red masked small yellow and orange square box", "Place the small yellow and orange square box in the box", "Place the flat yellow rectangular packet in the box", "Pick up the flat yellow rectangular packet", "Approach the red masked flat yellow rectangular packet", "Approach the red masked medium flat brown cardboard box with white round stickers", "Pick up the medium flat brown cardboard box with white round stickers", "Place the medium flat brown cardboard box with white round stickers in the box", "Pick up the black square box with white text (Dynamixel)", "Approach the red masked black square box with white text (Dynamixel)", "Place the black square box with white text (Dynamixel) in the box", "Place the yellow blister pack with a green and red lip balm in the box", "Approach the red masked yellow blister pack with a green and red lip balm", "Pick up the yellow blister pack with a green and red lip balm", "Pick up the green square snack packaging with brown accents", "Place the green square snack packaging with brown accents in the box", "Approach the red masked green square snack packaging with brown accents", "Pick up the pink and white tissue box with a bird graphic", "Approach the red masked pink and white tissue box with a bird graphic", "Place the pink and white tissue box with a bird graphic in the box", "Pick up the white tights packaging with a picture of legs", "Approach the red masked white tights packaging with a picture of legs", "Place the white tights packaging with a picture of legs in the box", "Approach the red masked small white box with purple accents", "Place the small white box with purple accents in the box", "Pick up the small white box with purple accents", "Approach the red masked small light blue and white rectangular box", "Place the small light blue and white rectangular box in the box", "Pick up the small light blue and white rectangular box", "Place the purple Skittles candy bag in the box", "Pick up the purple Skittles candy bag", "Approach the red masked purple Skittles candy bag", "Approach the red masked sD memory card in a white and green blister pack", "Pick up the sD memory card in a white and green blister pack", "Place the sD memory card in a white and green blister pack in the box", "Approach the red masked dark/Black patterned fabric in clear plastic", "Pick up the dark/Black patterned fabric in clear plastic", "Place the dark/Black patterned fabric in clear plastic in the box", "Approach the red masked long red rectangular packaging box", "Place the long red rectangular packaging box in the box", "Pick up the long red rectangular packaging box", "Place the set of markers in a transparent plastic pouch in the box", "Approach the red masked set of markers in a transparent plastic pouch", "Pick up the set of markers in a transparent plastic pouch", "Approach the red masked dark and white striped shirt in plastic", "Pick up the dark and white striped shirt in plastic", "Place the dark and white striped shirt in plastic in the box", "Place the clear medical syringe in white packaging in the box", "Pick up the clear medical syringe in white packaging", "Approach the red masked clear medical syringe in white packaging", "Approach the red masked coiled red cable tied with a white strap", "Place the coiled red cable tied with a white strap in the box", "Pick up the coiled red cable tied with a white strap", "Approach the red masked brown folded fabric in clear plastic", "Place the brown folded fabric in clear plastic in the box", "Pick up the brown folded fabric in clear plastic", "Pick up the white and green wall charger box", "Place the white and green wall charger box in the box", "Approach the red masked white and green wall charger box", "Place the orange cylindrical tub with a white lid in the box", "Pick up the orange cylindrical tub with a white lid", "Approach the red masked orange cylindrical tub with a white lid", "Place the long orange, black, and white wrapper in the box", "Approach the red masked long orange, black, and white wrapper", "Pick up the long orange, black, and white wrapper", "Approach the red masked white shirt with pink and green floral embroidery in plastic", "Pick up the white shirt with pink and green floral embroidery in plastic", "Place the white shirt with pink and green floral embroidery in plastic in the box", "Place the small gold and silver metal piece in a tiny plastic bag in the box", "Pick up the small gold and silver metal piece in a tiny plastic bag", "Approach the red masked small gold and silver metal piece in a tiny plastic bag", "Approach the red masked white seed packet with a picture of an orange papaya", "Pick up the white seed packet with a picture of an orange papaya", "Place the white seed packet with a picture of an orange papaya in the box", "Approach the red masked light blue and brown TOMS shoebox", "Place the light blue and brown TOMS shoebox in the box", "Pick up the light blue and brown TOMS shoebox", "Pick up the dark gray clothing in clear packaging with a white top label", "Approach the red masked dark gray clothing in clear packaging with a white top label", "Place the dark gray clothing in clear packaging with a white top label in the box", "Place the small colorful (blue, red, green) rectangular box in the box", "Approach the red masked small colorful (blue, red, green) rectangular box", "Pick up the small colorful (blue, red, green) rectangular box", "Pick up the large brown cardboard box with white sealing tape", "Place the large brown cardboard box with white sealing tape in the box", "Approach the red masked large brown cardboard box with white sealing tape", "Approach the red masked dark blue FILA shoebox with red and white logo", "Pick up the dark blue FILA shoebox with red and white logo", "Place the dark blue FILA shoebox with red and white logo in the box", "Place the white square plastic packet in the box", "Approach the red masked white square plastic packet", "Pick up the white square plastic packet", "Place the small plain brown rectangular box in the box", "Approach the red masked small plain brown rectangular box", "Pick up the small plain brown rectangular box", "Place the black gloves in clear plastic packaging in the box", "Pick up the black gloves in clear plastic packaging", "Approach the red masked black gloves in clear plastic packaging", "Place the tall yellow pencil box with blue and red text in the box", "Approach the red masked tall yellow pencil box with blue and red text", "Pick up the tall yellow pencil box with blue and red text", "Approach the red masked pink and white tissue box with a flower graphic", "Place the pink and white tissue box with a flower graphic in the box", "Pick up the pink and white tissue box with a flower graphic"
<a class="flex" href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=lets-merged/lets-human-mim-chess-vlas-ind-pnp-012-fourstage-masked--k24m50_lets-human-mim-chess-vlas-jr-pnp-12345-50p-fourstage-masked--k24m50--24019039">
<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/>
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/>
</a>
Training Configuration
| Setting | Value |
|---|
| Training steps | 25000 |
| Batch size | 16 |
| Optimizer | adamw |
| Learning rate | 2.5e-05 |
| Seed | 1000 |
| LeRobot version | 0.6.0 |
How to Get Started with the Model
New to LeRobot? These guides cover the full workflow:
- Install LeRobot — set up the
lerobot package.
- Hardware setup — assemble, wire, and calibrate your robot and cameras.
- Record data & train a policy — the end-to-end imitation-learning walkthrough.
- CLI cheat-sheet — quick reference for the
lerobot-* commands.
The short version to run and train this policy:
Run the policy on your robot
lerobot-rollout \
--strategy.type=base \
--robot.type=<your_robot_type> \
--robot.port=<your_robot_port> \
--robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \
--policy.path=mim-chess-vlas/jr-pnp-ab-50p-masked-fourstage__pi__05 \
--task="Place the white shirt with light blue stripes in plastic in the box" \
--duration=60
Replace the remaining <...> placeholders with your own values: --robot.port and the camera names/indices are specific to your machine, and the camera names must match the observation keys this policy was trained on.
When --strategy.type=base is used the script doesn't record the episodes. Skipping duration will make the policy run indefinitely. For more information look at rollout documentation.
Train your own policy
This policy type is usually fine-tuned from the pretrained base model lerobot/pi05_base:
lerobot-train \
--dataset.repo_id=${HF_USER}/<dataset> \
--policy.path=lerobot/pi05_base \
--output_dir=outputs/train/<policy_repo_id> \
--job_name=lerobot_training \
--policy.device=cuda \
--policy.repo_id=${HF_USER}/<policy_repo_id> \
--wandb.enable=true
Writes checkpoints to outputs/train/<policy_repo_id>/checkpoints/.
Evaluation
<!-- Report real-robot results here: run the policy several times per task and count the
successes. Delete the "No evaluation results" line and fill in this table instead:
| Task | Trials | Successes | Success rate |
| ---- | ------ | --------- | ------------ |
| pick the lego brick | 10 | 8 | 80% |
Also worth noting: anything that affects difficulty (new object positions, lighting,
distractors, a different robot of the same type, ...).
-->
No evaluation results have been provided for this policy yet.
Citation
If you use this policy, please cite the method linked in the description above, along with LeRobot:
@misc{cadene2024lerobot,
author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
howpublished = "\url{https://github.com/huggingface/lerobot}",
year = {2024}
}