pi05-relative-eef-bagplace-aug100k-full-ft-100k

pi0.5 — relative actions, end-effector space (13-dim: xyz + 6D rotation + gripper + base) — bag-place, image augmentation

Full fine-tune of lerobot/pi05_base on a single task — "pick up the bag on the ground and place it on the table" — with actions expressed relative to the observation state at the start of each chunk, and with photometric image augmentation enabled.

This is step 100K of a 100K-step run. Held-out loss here is 298% above the run's minimum, so these are not the best weights; they are published for step-matched comparison against the non-augmented run and the all-10 releases.

Held-out loss

5% of episodes are held out (10 of 200; 190 train).

The minimum is at 2K (0.0315). The curve rises after it: more steps make this model worse on held-out data, which is why the minimum rather than the endpoint is the checkpoint to prefer.

Augmentation

--dataset.image_transforms.enable=true, at most 3 of the following sampled per example. All are photometric; there is no geometric transform, so the image-to-action geometry the policy learns is untouched.

Comparison with the non-augmented run

There is a nearest counterpart — same data, same relative-action scheme, same init, same 2.5e-5 peak lr, same batch — run without augmentation. It is not a clean ablation, because the augmentation switch is not the only thing that differs:

The cosine horizons differ, so the two runs sit at different learning rates from warmup onward and the gap widens with step: by 20K this run's lr is 9.1x the other's. Any difference measured there (eef at 20K: 0.0518 here, 0.0728 there) therefore confounds augmentation with the schedule and should not be read as an augmentation effect.

The minima are more comparable: both arrive at 2K, where the two schedules are still close (lr differs by 2.2%), giving 0.0315 here against 0.0317 there. That is a ~1% gap from a single seed per arm — not enough to claim an effect in either direction.

No claim is made here that augmentation helped. Settling that would need the two runs at a matched decay horizon and more than one seed. The non-augmented run is not published.

Action representation

use_relative_actions=true with relative_exclude_joints=["gripper"]. This is a processor transform, not a data rewrite: RelativeActionsProcessorStep subtracts the anchor state at batch time and AbsoluteActionsProcessorStep adds it back on the way out, so the policy emits absolute actions and the conversion is identical in training, evaluation and inference.

action[t+k] -= observation.state[anchor]     for all dims EXCEPT gripper
gripper                                       absolute (it is a command, not a pose)

Action dim 13: eef_x, eef_y, eef_z, eef_xx, eef_xy, eef_xz, eef_yx, eef_yy, eef_yz, gripper, base_x, base_y, base_yaw — 12 of 13 dims relative, gripper absolute.

Normalisation is QUANTILES, computed from the relative offsets over chunks of 50 (meta/relative_action_provenance.json records this), not from the absolute targets. Absolute statistics would put most targets far outside [-1, 1].

The end-effector rotation is the stored continuous 6D form (first two rotation-matrix columns), not axis-angle. On this robot the gripper points down, so |rotvec| sits near pi where the sign flips arbitrarily; a built axis-angle view carries frame-to-frame jumps of 2*pi.

Training data

One language-conditioned task, 200 episodes, 477,058 frames (2.65 h at 50 fps), one u850 arm on a mobile base, three cameras (left / right / wrist).

bag-place is also ~7% of the all-10 mixture, so maskjp/pi05-relative-eef-all10-full-ft-100k has already trained on this task. Initialising from lerobot/pi05_base instead makes this a single-task specialist — the clean counterpart to that generalist.

Recipe

Loading

config.json here has had vision_encoder_lr_multiplier and state_in_prompt removed: both are training-only fields from the fork this was trained in, and stock LeRobot rejects unknown config keys. Inference is unaffected.

from lerobot.policies.pi05.modeling_pi05 import PI05Policy
policy = PI05Policy.from_pretrained("maskjp/pi05-relative-eef-bagplace-aug100k-full-ft-100k")

meta/ ships the dataset metadata for deployments that want it. Its info.json has per-dimension names flattened to a plain list; the training datasets store them grouped as {"motors": [...]}, which stock LeRobot resolves to the single key "motors" and which would silently relativise the gripper.

Related

• maskjp/pi05-relative-eef-bagplace-aug100k-full-ft-2k — the same arm at 2K.
• maskjp/pi05-relative-eef-all10-full-ft-100k — the all-10 generalist, which includes this task. Different training data; not a drop-in replacement.