pi05_xarm_dishwasher_binmarker9_lora

pi05 xArm dishwasher - binmarker9 (9 arrows/finger, contact bit only)

pi0.5 LoRA fine-tune for the xArm dishwasher task ("pull the basket outside of the dishwasher and pick up the mug and put it into the basket"), trained on frames with the binmarker9 tactile overlay burned in.

What the overlay encodes

Only the per-finger binary contact bit. Both force direction and force magnitude are omitted:

9 arrows per finger (one at each of the 9 forward-kinematics-projected sensor pad cells), drawn iff that finger's contact state is ON (max-cell metric, hysteresis ON >= 0.25 / OFF < 0.18, 2-on/3-off debounce, state reset per episode).

Verified tactile-independence: 0 max pixel change across 300 randomized force readings at a fixed pose - the pixels provably carry no force information.

Ablation pair

Hyperparameters are identical across both arms (LoRA gemma_2b_lora + gemma_300m_lora, batch size 8, cosine schedule warmup 500 / peak 1e-4 / decay 20k to 1e-5, AdamW grad-clip 1.0, no EMA, 20k max steps with early stopping), so differences are attributable to the overlay alone. Trained 13100 steps.

Contents

Orbax checkpoint (params/) plus assets/ norm stats - inference-ready. train_state/ (optimizer state) is intentionally excluded.

Inference

python examples/xarm/inference/run_xarm_inference.py \
  --checkpoint <local-path> \
  --overlay binmarker9_dishwasher

The overlay must be redrawn at deployment exactly as in training, or the policy is out of distribution. The inference renderer (FixedMarkerOverlay, variant binmarker9) is verified byte-exact against the training renderer: 0 max pixel error over 3,566 camera images across all 4 tasks (cube 450, tube 760, charger 1,122, dishwasher 1,234), 0 contact-state mismatches. The --overlay flag requires the per-task suffix because the per-finger p99 scales differ up to ~6x across tasks.

Observation/action convention: state 8-dim [ee_pos_m(3), ee_axis_angle_rad(3), grasp, grasp], action 7-dim [dxyz_m, d_axis_angle_rad, grasp_pm1], 224x224 RGB uint8, 10 Hz, action chunk 10 (consume 5).

Paired dataset

EdwardoSunny/xarm_dishwasher_binmarker9