Downloads · 30 days
31
34% of all-time downloads
nepyope/pi05-can-to-martino-12k
pi05-can-to-martino-12k is a robotics model from nepyope. Use it for the robotics task on the model card, and read the license before you ship it in a product. It is set up for lerobot.
π₀.₅ fine-tuned from lerobot/pi05base on whole-body Unitree G1 teleoperation data. Standard flow-matching objective — no real-time chunking, no πR² staircase — trained on upstream main.
Downloads · 30 days
31
34% of all-time downloads
All-time downloads
90
Public
Parameters
4.1B
16.6 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors16.6 GB · 100%
From the Hugging Face model README
π₀.₅ fine-tuned from lerobot/pi05_base on whole-body Unitree G1 teleoperation data. Standard
flow-matching objective — no real-time chunking, no πR² staircase — trained on upstream main.
Successor to nepyope/pi05-can-to-martino-8k:
2.2× the data (105 episodes vs 44), 12000 steps instead of 8000, and a different task string
(Bring the can to the white table, not Move the can from the low table to Martino). Same
hyperparameters otherwise.
| base checkpoint | lerobot/pi05_base |
| dataset | nepyope/can_clean_final @ 8ef217671b2286d5103736fba3f7c7bc60f5756d — 105 episodes, 212,290 frames, 50 fps |
| robot | unitree_g1, Damiao CAN grippers on both hands |
| cameras | 3: ego_view, left_wrist, right_wrist at 480×640 |
| task | Bring the can to the white table |
| action dim | 66 = 64 joints + 2 grippers |
| state dim | 31 = 29 DOF + 2 grippers, padded to max_state_dim=32 |
chunk_size | 50 — at 50 fps this is 1.0 s of motion |
| trainable params | 693M of 4.14B (train_expert_only=true, VLM frozen) |
| hardware | 4×H100 80GB, one node, 35.4 GB per GPU |
| batch | 32 per GPU × 4 = 128, no gradient accumulation |
| optimizer | AdamW, LR 1e-4, weight decay 1e-4, betas (0.9, 0.95), grad clip 10.0, cosine decay to 1e-5 with 500 warm-up steps over 12000 |
| throughput | 1.85 s/step, 71 samples/s — 12000 steps in 6h09m (7.24 epochs) |
| code | lerobot main @ 8bf6056d1 |
| step | 1k | 2k | 3k | 4k | 5k | 6k | 7k | 8k | 9k | 10k | 11k | 12k |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| epoch | 0.60 | 1.21 | 1.81 | 2.41 | 3.01 | 3.62 | 4.22 | 4.82 | 5.43 | 6.03 | 6.63 | 7.24 |
| loss | 0.077 | 0.055 | 0.046 | 0.041 | 0.037 | 0.032 | 0.030 | 0.030 | 0.032 | 0.026 | 0.025 | 0.025 |
| grad norm | 0.221 | 0.096 | 0.071 | 0.062 | 0.054 | 0.050 | 0.045 | 0.044 | 0.044 | 0.039 | 0.037 | 0.038 |
Starts at 1.248 (step 50) and is under 0.08 within one epoch. Flat from step 10k with the cosine fully annealed, so this is converged rather than truncated. Final 0.025 is marginally below the 44-episode run's 0.031 despite 2.4× fewer epochs, which is what more data should do. This is training loss with no held-out split; only a closed-loop rollout settles quality.
--fps=50 is not optional. The policy has no notion of frame rate; it only learned the 20 ms
action spacing present in the data. At --fps=30 it executes the motion 1.67× slower than
demonstrated.
The task string must match the one above verbatim — pi0.5 conditions on it as text, and this checkpoint saw exactly one task during training.
A synchronous sample_actions call costs ~272 ms, which at 50 fps is 13.6 control ticks — so
synchronous inference stalls the robot between chunks. RTC's guided mode hides that latency and works
with any pi0.5 checkpoint, including this one:
lerobot-rollout \
--strategy.type=base \
--policy.path=nepyope/pi05-can-to-martino-12k \
--inference.type=rtc \
--inference.rtc.mode=guided \
--inference.rtc.execution_horizon=20 \
--inference.queue_threshold=20 \
--robot.type=unitree_g1 \
--task="Bring the can to the white table" \
--fps=50
execution_horizon=20 is derived, not arbitrary: one call takes ~13.6 ticks at 50 fps, so the
horizon has to exceed that or the next chunk arrives after the current one has been consumed. 20
ticks is 400 ms of actions per call, leaving ~45% headroom. The library default of 10 would be 200 ms
and would consistently overrun at this frame rate. Raise it if calls overrun; lower it for
reactivity, but not below ~15.
lerobot-rollout \
--strategy.type=base \
--policy.path=nepyope/pi05-can-to-martino-12k \
--inference.type=sync \
--robot.type=unitree_g1 \
--task="Bring the can to the white table" \
--fps=50
Fine for a first smoke test on hardware, but expect a visible pause every chunk.
--inference.type=pir2 — the πR² engine refuses anything whose config is not
rtc_training_schedule: staircase. This one was trained on main, which has no such field.--inference.rtc.mode=trained — needs a checkpoint trained with rtc_training_max_delay > 0.Both require a differently-trained model, not a flag change. Use
nepyope/pi05-tshirt-staircase-8k if you
want to exercise the πR² path.
accelerate launch --num_processes=4 --mixed_precision=bf16 \
-m lerobot.scripts.lerobot_train \
--policy.type=pi05 \
--policy.pretrained_path=lerobot/pi05_base \
--policy.max_state_dim=32 --policy.max_action_dim=66 \
--policy.train_expert_only=true \
--policy.freeze_vision_encoder=false \
--policy.gradient_checkpointing=true \
--policy.compile_model=false \
--policy.push_to_hub=false \
--policy.chunk_size=50 --policy.n_action_steps=50 \
--dataset.repo_id=nepyope/can_clean_final \
--dataset.root=/path/to/can_clean_final \
--dataset.use_imagenet_stats=false \
--batch_size=32 --num_workers=10 --steps=12000 \
--tolerance_s=0.001 \
--use_policy_training_preset=false \
--optimizer.type=adamw --optimizer.lr=1e-4 --optimizer.weight_decay=1e-4 \
--optimizer.betas="[0.9,0.95]" --optimizer.grad_clip_norm=10.0 \
--scheduler.type=cosine_decay_with_warmup \
--scheduler.num_warmup_steps=500 --scheduler.num_decay_steps=12000 \
--scheduler.peak_lr=1e-4 --scheduler.decay_lr=1e-5 \
--wandb.enable=true --wandb.project=can-clean-final --wandb.disable_artifact=true \
--save_freq=2000 --log_freq=50 \
--output_dir=/path/to/output --job_name=can_clean_12k
1. A patch to from_pretrained (required — without it you train from scratch).
max_action_dim=66 disagrees with pi05_base's 32, so action_in_proj / action_out_proj cannot be
loaded. In PI05Policy.from_pretrained the resulting load_state_dict error is swallowed by a broad
except, which returns a randomly initialized model that then trains and logs perfectly
normally. A local patch drops only the shape-mismatched tensors so the backbone loads and just those
two projections start fresh:
Dropping 3 shape-mismatched keys (re-initialized):
- model.action_in_proj.weight: ckpt (1024, 32) vs model (1024, 66)
- model.action_out_proj.bias: ckpt (32,) vs model (66,)
- model.action_out_proj.weight: ckpt (32, 1024) vs model (66, 1024)
num_learnable_params=693M of 4.14B confirms the frozen VLM did load. Anyone reproducing this on
upstream main should verify those lines appear — without them the run silently trains from scratch.
2. --dataset.use_imagenet_stats=false. make_dataset in src/lerobot/datasets/factory.py
assumes every camera key has an entry in meta/stats.json and raises
KeyError: 'observation.images.ego_view' otherwise. This dataset's stats cover only action,
observation.state and the index columns. Harmless to disable: pi0.5's VISUAL normalization is
IDENTITY, so image statistics are never read.
3. --tolerance_s=0.001. Default is 1e-4, and decoding hit
FrameTimestampError: queried 1030.5599 vs loaded 1030.5601. A 0.2 ms gap is one hundredth of a
frame interval at 50 fps — float error accumulating over a 17-minute video chunk, not a real
desynchronisation. 0.001 is still 5% of a frame, far too small to fetch a wrong frame.
--dataset.root bypasses Hub revision resolution and guarantees training reads 8ef217671. Unlike
some earlier G1 datasets, this one already ships q01/q99 quantile stats for state and action,
which pi0.5 requires for QUANTILES normalization, so no augment_dataset_quantile_stats.py pass
was needed.
updt_s, data_s and mem_gb in this run's logs read 7.17, 0.23 and 141.55 — each exactly 4× too
large, and smp/s 4× too small. MetricsTracker.reduce_across_ranks asks accelerate for a max
reduction, which it does not implement, so the values are summed over the 4 ranks instead. The real
figures are the ones in the table above (1.85 s/step, 71 samples/s, 35.4 GB/GPU), confirmed against
wall clock between log lines.