pi05-dexverse

π₀.₅ DexVerse baseline checkpoints

π₀.₅ policies finetuned on the DexVerse tabletop dexterous-manipulation benchmark, for use with the online evaluation harness in scripts/eval/.

Two embodiments of a floating Shadow hand are covered:

The state is a 3-frame stack (3 × 28 and 3 × 56) — this is what the DexVerse rgb observation preset produces and what the norm stats below were computed over. eval_policy.py --policy pi0 applies that preset by default.

Download

# single-hand
hf download dexverse/pi05-dexverse --include 'single/*' --local-dir ./pi05-ckpt
# bimanual
hf download dexverse/pi05-dexverse --include 'bimanual/*' --local-dir ./pi05-ckpt

CKPT_DIR for serve_pi0.sh is then ./pi05-ckpt/single (or bimanual).

Contents

single/
├── model.safetensors                                        # 7.5 GB
├── metadata.pt
└── assets/dexbench-data/lerobot-single/norm_stats.json

Do not move or rename the assets/ subtree. openpi resolves normalization statistics by assets/<asset_id>/norm_stats.json, where asset_id comes from the training config's data config. The historical id is dexbench-data/... (DexVerse was previously named DexBench); renaming it makes the server start without norm stats and silently emit unnormalized actions.

Serving

The policy runs out of process — its JAX/torch pins are not compatible with Isaac Sim's. From a DexVerse checkout:

OPENPI_ROOT=/path/to/openpi \
PYTHON_BIN=/path/to/openpi/.venv/bin/python \
CKPT_DIR=$PWD/pi05-ckpt/single \
CONFIG_NAME=pi05_dexbench \
    bash scripts/eval/serve_pi0.sh

# in the DexVerse / Isaac Lab environment
python scripts/eval/eval_policy.py --policy pi0 --enable_cameras --headless \
    --task Dexverse-GraspCup-v0 --num_episodes 20

CONFIG_NAME must be a TrainConfig registered in your openpi checkout whose data config matches the embodiment (the bimanual config repacks three camera views instead of two).

The wire format the server expects — produced by eval_policy.py:

{"third_person": uint8 (256,256,3),
 "wrist": uint8 (256,256,3),        # left_wrist + right_wrist when bimanual
 "state": float32 (84,),            # (168,) when bimanual
 "prompt": str}                     # -> {"actions": float32 (horizon, 28|56)}

The prompt strings are part of the benchmark definition and are listed in scripts/eval/baseline_tasks.py; they are the training set's language_instruction values verbatim. The data config sets prompt_from_task=True, so the client-sent string reaches the model directly.

Training

Finetuned from pi05_base for 2000 steps on demonstrations recorded through the DexVerse VR teleoperation pipeline, converted to LeRobot format. Backgrounds and table textures were held fixed during data generation (create_demo_files_sequential.py --disable-bg-randomize); evaluate with the matching setting (the harness default) or expect a visual domain gap.

License

Derived from π₀.₅, whose VLM backbone is PaliGemma — these weights are therefore subject to the Gemma Terms of Use. The openpi and DexVerse code is separately licensed (Apache-2.0 and BSD-3-Clause respectively).