pi0.5-libero-plus-lora

pi0.5 LIBERO-plus LoRA

LoRA fine-tuned weights of pi0.5 (PaliGemma-2B + Action Expert 300M) on the LIBERO-plus dataset, targeting the Action Expert and projection layers.

---

1. Model Configuration

---

2. Training

Dataset

The model was trained on nosuke113/libero-plus-deduplicated, which is a deduplicated version of the original lerobot/libero_plus dataset.

Data Split Methodology

The validation set was carefully constructed to prevent data leakage. Since the original dataset contains identical physical trajectories artificially augmented with different visual appearances, a simple random split would result in the model evaluating on physical trajectories it has already seen during training, leading to an artificially low validation loss.

To address this, we computed a SHA-1 hash over the full action sequence of each episode. Episodes sharing the same action hash (i.e., identical physical trajectories) were grouped together. The train/val split was then performed at the trajectory level, ensuring that all episodes belonging to a specific physical trajectory are placed entirely in either the training set or the validation set.

This strict trajectory-level separation means the model's validation loss accurately reflects its ability to generalize to unseen physical demonstrations, rather than just memorizing seen trajectories with different textures or lighting.

Training Details

---

3. Evaluation Dataset & Perturbation Axes

The evaluation dataset containing the 180 held-out tasks is available at nosuke113/libero-plus-evaluation. These tasks are constructed along 4 perturbation axes from the LIBERO-plus BDDL scene definitions.

1. Base — Standard BDDL scenes. Camera (0, 0, 100, 0, 0), fixed object poses, no noise.
2. View — Camera orbit angle (0-356 deg), zoom distance (100-191), and gaze tilt are varied.
3. Noise — Image corruptions: Gaussian noise, motion blur, fog, glass distortion (level 8-48).
4. Moved — Object initial positions shifted by several cm from default.

---

4. Evaluation Results (900 Rollouts)

Measured on 180 fully held-out tasks with n=5 seeds (900 total rollouts). Closed-loop control with max 600 steps per episode, bfloat16 mixed precision on NVIDIA L4 GPU. Inference used Receding Horizon Control (RTC) with an action chunk size of 5. Note: The evaluation results below were measured using the checkpoint at 35,000 steps.

Breakdown by Perturbation

<br/> <div align="center"> <h4>Robustness across Perturbations</h4> <p><em>Parallel playback comparing Base, View, Noise, and Position perturbed evaluation environments.</em></p> <table width="100%"> <tr> <td width="50%" align="center" valign="top"> <video src="https://huggingface.co/nosuke113/pi0.5-libero-plus-lora/resolve/main/videos/grid_0.mp4" controls autoplay loop muted width="100%"></video> <br/><b>Task: Put the black bowl on the plate</b> </td> <td width="50%" align="center" valign="top"> <video src="https://huggingface.co/nosuke113/pi0.5-libero-plus-lora/resolve/main/videos/grid_1.mp4" controls autoplay loop muted width="100%"></video> <br/><b>Task: Turn on the stove bottom</b> </td> </tr> <tr> <td width="50%" align="center" valign="top"> <video src="https://huggingface.co/nosuke113/pi0.5-libero-plus-lora/resolve/main/videos/grid_2.mp4" controls autoplay loop muted width="100%"></video> <br/><b>Task: Put the black bowl in the top drawer</b> </td> <td width="50%" align="center" valign="top"> </td> </tr> </table> </div>

---

5. Quickstart

This model is fully compatible with the Hugging Face lerobot library. You can instantiate it with a few lines of code:

from lerobot.common.policies.factory import make_policy

# Load the policy with LoRA weights
policy = make_policy(repo_id="nosuke113/pi0.5-libero-plus-lora")
policy.eval()

# Example usage (assuming `obs` is provided by the environment)
# obs = {
#     "observation.images.agentview": ...,
#     "observation.images.robot0_eye_in_hand": ...,
#     "observation.state": ...,
# }
# action = policy.select_action(obs)

---

6. License & Attribution