shell-game-policies

shell_game — policies and planner

A cup-and-ball shuffle on a real robot. A red block is covered by one of three identical cups, the cups are shuffled by hand, and the arm has to lift the cup the block ended under and drop the block in a basket. At the moment of decision every cup looks the same, so the answer exists only in frames that have already gone past. The task was built as an out-of-simulator test of memory conditioning.

Dataset: 45 episodes, 18,442 frames, 7-DoF single arm plus gripper, 3 stereo cameras, 9.26 Hz measured control rate (the mp4 container claims 10 and is wrong).

pi0.5 policies

All three: pi05_base finetuned, 224x224, global batch 64, 10,000 steps = 34.7 epochs over 18,442 frames, 4xH100, final loss 0.0007.

rot6d rather than rpy: roll wraps at +-pi and produced 269 discontinuities across the 45 episodes; unwrapping and converting to rot6d leaves 0 (std 3.08 -> 0.039). The delta mask is (9, -1) — position and rotation relative to the current state, gripper absolute.

Each directory holds params/ and assets/ for the final step. train_state/ (optimizer moments, 31 GB per step) is not uploaded: these load for inference, not to resume training. assets/ carries the normalisation statistics and is required for correct actions.

HLP planner (hlp-lora/)

LoRA r16/alpha32 over stock Qwen3-VL-4B-Instruct, trained on 1,171 Local-STTP rows built from the wrist camera. The contract: the prompt carries the subtask the controller last accepted, and the model emits 1 or 0 for advance/hold, a newline, then {"current_subtask": ..., "keyframe_positions": [...]}. When the first token is 0 the subtask field holds the action after the accepted one, with unpredictable reserved for the case where it cannot yet be determined — during the shuffle that field is exactly "which cup will be uncovered".

Load the base model from Qwen/Qwen3-VL-4B-Instruct and apply this adapter.

What is known about how well these work

Nothing yet for the pi0.5 policies: there is no evaluation harness for this environment, and the offline protocol is still being built.

For the planning question, zero-shot numbers on the same 45 episodes exist and are sobering:

grounding asks which of the three cups the arm is lifting, from frames where it is already lifting one — the answer is in the picture. shuffle tracking asks which cup hides the block after the shuffle, which needs the frames that have scrolled past. Perception is solved and tracking through occlusion is at chance. The larger model is perfect at grounding and worse at tracking, so this is not an axis that scale fixes.

Baselines matter here: always answering "left" scores 40.0 % and lifting the cup the block started under scores 33.3 %, so 40 % is the floor, not zero. n = 45.