Model Card for pi05
<!-- Provide a quick summary of what the model is/does. -->
π₀.₅ (Pi05) is a Vision-Language-Action model from Physical Intelligence designed for open-world generalization: it evolves π₀ to generalize to entirely new environments and situations that were never seen during training. The LeRobot implementation is adapted from their open-source OpenPI repository.
<!-- A short demo is worth more than any description! Record a GIF/video of the policy
running on your robot, upload it to this repo, and embed it here:
<p align="center">
<img src="https://huggingface.co/<hf_user>/<policy_repo_id>/resolve/main/demo.gif" width="60%"/>
</p>
-->
This policy has been trained and pushed to the Hub using LeRobot.
Learn how to train and run it in the LeRobot pi05 guide, or browse the full documentation.
Model Details
- License: apache-2.0
- Fine-tuned from: lerobot/pi05_base
- Cameras:
side, wrist_left, wrist_right
Inputs & Outputs
The policy consumes these observation features and produces these action features.
Inputs
| Feature | Type | Shape |
|---|
observation.images.side | VISUAL | (3, 300, 355) |
observation.images.wrist_left | VISUAL | (3, 480, 848) |
observation.images.wrist_right | VISUAL | (3, 480, 848) |
observation.state | STATE | (7,) |
Outputs
| Feature | Type | Shape |
|---|
action | ACTION | (7,) |
Training Dataset
- Repository: lets-merged/lets-human-mim-chess-vlas-ind-pnp-012-fourstage--k24m50_lets-human-mim-chess-vlas-jr-pnp-12345-30p-fourstage--k24m50--24019030
- Episodes: 924
- Frames: 515161
- Frame rate: 50 FPS
- Task(s): "Place the white shirt with light blue stripes in plastic in the box", "Leave the box", "Approach the white shirt with light blue stripes in plastic", "Pick up the white shirt with light blue stripes in plastic", "Pick up the small white square box with black text", "Approach the small white square box with black text", "Place the small white square box with black text in the box", "Approach the small solid yellow cube", "Pick up the small solid yellow cube", "Place the small solid yellow cube in the box", "Pick up the small pink round container", "Place the small pink round container in the box", "Approach the small pink round container", "Pick up the small yellow and orange square box", "Place the small yellow and orange square box in the box", "Approach the small yellow and orange square box", "Place the flat yellow rectangular packet in the box", "Approach the flat yellow rectangular packet", "Pick up the flat yellow rectangular packet", "Pick up the medium flat brown cardboard box with white round stickers", "Approach the medium flat brown cardboard box with white round stickers", "Place the medium flat brown cardboard box with white round stickers in the box", "Approach the black square box with white text (Dynamixel)", "Place the black square box with white text (Dynamixel) in the box", "Pick up the black square box with white text (Dynamixel)", "Place the yellow blister pack with a green and red lip balm in the box", "Pick up the yellow blister pack with a green and red lip balm", "Approach the yellow blister pack with a green and red lip balm", "Place the green square snack packaging with brown accents in the box", "Pick up the green square snack packaging with brown accents", "Approach the green square snack packaging with brown accents", "Place the pink and white tissue box with a bird graphic in the box", "Approach the pink and white tissue box with a bird graphic", "Pick up the pink and white tissue box with a bird graphic", "Approach the white tights packaging with a picture of legs", "Place the white tights packaging with a picture of legs in the box", "Pick up the white tights packaging with a picture of legs", "Approach the small white box with purple accents", "Place the small white box with purple accents in the box", "Pick up the small white box with purple accents", "Approach the small light blue and white rectangular box", "Place the small light blue and white rectangular box in the box", "Pick up the small light blue and white rectangular box", "Approach the purple Skittles candy bag", "Place the purple Skittles candy bag in the box", "Pick up the purple Skittles candy bag", "Pick up the sD memory card in a white and green blister pack", "Place the sD memory card in a white and green blister pack in the box", "Approach the sD memory card in a white and green blister pack", "Pick up the dark/Black patterned fabric in clear plastic", "Place the dark/Black patterned fabric in clear plastic in the box", "Approach the dark/Black patterned fabric in clear plastic", "Approach the long red rectangular packaging box", "Pick up the long red rectangular packaging box", "Place the long red rectangular packaging box in the box", "Place the set of markers in a transparent plastic pouch in the box", "Approach the set of markers in a transparent plastic pouch", "Pick up the set of markers in a transparent plastic pouch", "Approach the dark and white striped shirt in plastic", "Pick up the dark and white striped shirt in plastic", "Place the dark and white striped shirt in plastic in the box", "Place the clear medical syringe in white packaging in the box", "Approach the clear medical syringe in white packaging", "Pick up the clear medical syringe in white packaging", "Pick up the coiled red cable tied with a white strap", "Approach the coiled red cable tied with a white strap", "Place the coiled red cable tied with a white strap in the box", "Place the brown folded fabric in clear plastic in the box", "Approach the brown folded fabric in clear plastic", "Pick up the brown folded fabric in clear plastic", "Place the white and green wall charger box in the box", "Approach the white and green wall charger box", "Pick up the white and green wall charger box", "Approach the orange cylindrical tub with a white lid", "Place the orange cylindrical tub with a white lid in the box", "Pick up the orange cylindrical tub with a white lid", "Pick up the long orange, black, and white wrapper", "Approach the long orange, black, and white wrapper", "Place the long orange, black, and white wrapper in the box", "Place the white shirt with pink and green floral embroidery in plastic in the box", "Pick up the white shirt with pink and green floral embroidery in plastic", "Approach the white shirt with pink and green floral embroidery in plastic", "Approach the small gold and silver metal piece in a tiny plastic bag", "Pick up the small gold and silver metal piece in a tiny plastic bag", "Place the small gold and silver metal piece in a tiny plastic bag in the box", "Place the white seed packet with a picture of an orange papaya in the box", "Approach the white seed packet with a picture of an orange papaya", "Pick up the white seed packet with a picture of an orange papaya", "Pick up the light blue and brown TOMS shoebox", "Approach the light blue and brown TOMS shoebox", "Place the light blue and brown TOMS shoebox in the box", "Pick up the dark gray clothing in clear packaging with a white top label", "Place the dark gray clothing in clear packaging with a white top label in the box", "Approach the dark gray clothing in clear packaging with a white top label", "Pick up the small colorful (blue, red, green) rectangular box", "Approach the small colorful (blue, red, green) rectangular box", "Place the small colorful (blue, red, green) rectangular box in the box", "Place the large brown cardboard box with white sealing tape in the box", "Approach the large brown cardboard box with white sealing tape", "Pick up the large brown cardboard box with white sealing tape", "Pick up the dark blue FILA shoebox with red and white logo", "Approach the dark blue FILA shoebox with red and white logo", "Place the dark blue FILA shoebox with red and white logo in the box", "Approach the white square plastic packet", "Pick up the white square plastic packet", "Place the white square plastic packet in the box", "Place the small plain brown rectangular box in the box", "Pick up the small plain brown rectangular box", "Approach the small plain brown rectangular box", "Pick up the black gloves in clear plastic packaging", "Approach the black gloves in clear plastic packaging", "Place the black gloves in clear plastic packaging in the box", "Pick up the tall yellow pencil box with blue and red text", "Approach the tall yellow pencil box with blue and red text", "Place the tall yellow pencil box with blue and red text in the box", "Approach the pink and white tissue box with a flower graphic", "Place the pink and white tissue box with a flower graphic in the box", "Pick up the pink and white tissue box with a flower graphic"
<a class="flex" href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=lets-merged/lets-human-mim-chess-vlas-ind-pnp-012-fourstage--k24m50_lets-human-mim-chess-vlas-jr-pnp-12345-30p-fourstage--k24m50--24019030">
<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/>
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/>
</a>
Training Configuration
| Setting | Value |
|---|
| Training steps | 25000 |
| Batch size | 16 |
| Optimizer | adamw |
| Learning rate | 2.5e-05 |
| Seed | 1000 |
| LeRobot version | 0.6.0 |
How to Get Started with the Model
New to LeRobot? These guides cover the full workflow:
- Install LeRobot — set up the
lerobot package.
- Hardware setup — assemble, wire, and calibrate your robot and cameras.
- Record data & train a policy — the end-to-end imitation-learning walkthrough.
- CLI cheat-sheet — quick reference for the
lerobot-* commands.
The short version to run and train this policy:
Run the policy on your robot
lerobot-rollout \
--strategy.type=base \
--robot.type=<your_robot_type> \
--robot.port=<your_robot_port> \
--robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \
--policy.path=mim-chess-vlas/jr-pnp-ab-30p-nomask-fourstage__pi__05 \
--task="Place the white shirt with light blue stripes in plastic in the box" \
--duration=60
Replace the remaining <...> placeholders with your own values: --robot.port and the camera names/indices are specific to your machine, and the camera names must match the observation keys this policy was trained on.
When --strategy.type=base is used the script doesn't record the episodes. Skipping duration will make the policy run indefinitely. For more information look at rollout documentation.
Train your own policy
This policy type is usually fine-tuned from the pretrained base model lerobot/pi05_base:
lerobot-train \
--dataset.repo_id=${HF_USER}/<dataset> \
--policy.path=lerobot/pi05_base \
--output_dir=outputs/train/<policy_repo_id> \
--job_name=lerobot_training \
--policy.device=cuda \
--policy.repo_id=${HF_USER}/<policy_repo_id> \
--wandb.enable=true
Writes checkpoints to outputs/train/<policy_repo_id>/checkpoints/.
Evaluation
<!-- Report real-robot results here: run the policy several times per task and count the
successes. Delete the "No evaluation results" line and fill in this table instead:
| Task | Trials | Successes | Success rate |
| ---- | ------ | --------- | ------------ |
| pick the lego brick | 10 | 8 | 80% |
Also worth noting: anything that affects difficulty (new object positions, lighting,
distractors, a different robot of the same type, ...).
-->
No evaluation results have been provided for this policy yet.
Citation
If you use this policy, please cite the method linked in the description above, along with LeRobot:
@misc{cadene2024lerobot,
author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
howpublished = "\url{https://github.com/huggingface/lerobot}",
year = {2024}
}