pi05-relative-eef-all10-full-ft-80k

pi0.5 — relative actions, end-effector space (13-dim: xyz + 6D rotation + gripper + base)

Full fine-tune of lerobot/pi05_base on all ten L5vel/*-eef-merged-v30 datasets, with actions expressed relative to the observation state at the start of each chunk.

This is step 80K of a 100K-step run. Held-out loss here is 66% above the run's minimum, so these are not the best weights; they are published for step-matched comparison against the previous 30K round.

Held-out loss

5% of episodes per task are held out (77 of 1499 episodes, all 11 tasks).

The minimum is at 10K (0.0245). The curve rises after it: more steps make this model worse on held-out data, which is why the minimum rather than the endpoint is the checkpoint to prefer.

Action representation

use_relative_actions=true with relative_exclude_joints=["gripper"]. This is a processor transform, not a data rewrite: RelativeActionsProcessorStep subtracts the anchor state at batch time and AbsoluteActionsProcessorStep adds it back on the way out, so the policy emits absolute actions and the conversion is identical in training, evaluation and inference.

action[t+k] -= observation.state[anchor]     for all dims EXCEPT gripper
gripper                                       absolute (it is a command, not a pose)

Action dim 13: eef_x, eef_y, eef_z, eef_xx, eef_xy, eef_xz, eef_yx, eef_yy, eef_yz, gripper, base_x, base_y, base_yaw — 12 of 13 dims relative, gripper absolute.

Normalisation is QUANTILES, computed from the relative offsets over chunks of 50 (meta/relative_action_provenance.json records this), not from the absolute targets. Absolute statistics would put most targets far outside [-1, 1].

The end-effector rotation is the stored continuous 6D form (first two rotation-matrix columns), not axis-angle. On this robot the gripper points down, so |rotvec| sits near pi where the sign flips arbitrarily; a built axis-angle view carries frame-to-frame jumps of 2*pi.

Training data

11 language-conditioned tasks, 1499 episodes, 3,089,476 frames (17.2 h at 50 fps), one u850 arm on a mobile base, three cameras (left / right / wrist).

The mixture is skewed: the four largest tasks are ~82% of frames. Sampling is uniform over frames, so it is not rebalanced.

Recipe

Loading

config.json here has had vision_encoder_lr_multiplier and state_in_prompt removed: both are training-only fields from the fork this was trained in, and stock LeRobot rejects unknown config keys. Inference is unaffected.

from lerobot.policies.pi05.modeling_pi05 import PI05Policy
policy = PI05Policy.from_pretrained("maskjp/pi05-relative-eef-all10-full-ft-80k")

meta/ ships the dataset metadata (~2 MB) for deployments that want it. Its info.json has per-dimension names flattened to a plain list; the training datasets store them grouped as {"motors": [...]}, which stock LeRobot resolves to the single key "motors" and which would silently relativise the gripper.

Related

• maskjp/pi05-relative-eef-all10-full-ft-10k — the same arm at 10K.
• maskjp/pi05-relative-eef-full-ft-30k — the previous round, 7 datasets / 949 episodes. Different training data; not a drop-in replacement.