pi05_xarm_cube_binmarker9_lora

pi05 xArm cube - binmarker9 (9 arrows/finger, contact bit only)

pi0.5 LoRA fine-tune for the xArm cube task ("pick up the cube"), trained on frames with the binmarker9 tactile overlay burned in.

What the overlay encodes

Only the per-finger binary contact bit. Both force direction and force magnitude are omitted:

9 arrows per finger (one at each of the 9 forward-kinematics-projected sensor pad cells), drawn iff that finger's contact state is ON (max-cell metric, hysteresis ON >= 0.25 / OFF < 0.18, 2-on/3-off debounce, state reset per episode).

Verified tactile-independence: 0 max pixel change across 300 randomized force readings at a fixed pose - the pixels provably carry no force information.

Ablation pair

Hyperparameters are identical across both arms (LoRA gemma_2b_lora + gemma_300m_lora, batch size 8, cosine schedule warmup 500 / peak 1e-4 / decay 20k to 1e-5, AdamW grad-clip 1.0, no EMA, 20k max steps with early stopping), so differences are attributable to the overlay alone. Trained 7900 steps.

Contents

Orbax checkpoint (params/) plus assets/ norm stats - inference-ready. train_state/ (optimizer state) is intentionally excluded.

Inference

python examples/xarm/inference/run_xarm_inference.py \
  --checkpoint <local-path> \
  --overlay binmarker9_cube

The overlay must be redrawn at deployment exactly as in training, or the policy is out of distribution. The inference renderer (FixedMarkerOverlay, variant binmarker9) is verified byte-exact against the training renderer: 0 max pixel error over 3,566 camera images across all 4 tasks (cube 450, tube 760, charger 1,122, dishwasher 1,234), 0 contact-state mismatches. The --overlay flag requires the per-task suffix because the per-finger p99 scales differ up to ~6x across tasks.

Observation/action convention: state 8-dim [ee_pos_m(3), ee_axis_angle_rad(3), grasp, grasp], action 7-dim [dxyz_m, d_axis_angle_rad, grasp_pm1], 224x224 RGB uint8, 10 Hz, action chunk 10 (consume 5).

Paired dataset

EdwardoSunny/xarm_cube_binmarker9