MolmoAct2-BimanualYAM-oranges-20k

MolmoAct2-BimanualYAM — "Put all oranges in the bowl" (20k steps: extended schedule + upstream training fixes)

Fine-tune of allenai/MolmoAct2-BimanualYAM on brandonyang/yam-vive-teleop (80 teleop episodes, 74,927 frames, bimanual YAM, 30 fps).

Successor to ASethi04/MolmoAct2-BimanualYAM-oranges-12k: same recipe (20,000 steps ≈ 17.1 epochs, LoRA r=64 on the VLM @ 5e-5, action expert fully fine-tuned @ 1e-4, narrowed color jitter with affine augmentation kept), retrained with the MolmoAct2 author's training fixes from huggingface/lerobot PR#4249:

• flow-matching noise/timesteps/velocity targets kept in fp32 under autocast (previously quantized to bf16)
• distributed-correct weighted-loss normalization (denominator all-reduced across ranks)
• episode-boundary actions supervised on clamped fixed-horizon targets instead of masked out — the policy now learns terminal settling behavior
• input-embedding freeze no longer risks freezing tied output parameters
• llm_residual_dropout=0.1 (original recipe's regularization)

Also new in this checkpoint: inference_action_mode="continuous" is baked into the config — no override needed at serving time.

Final training step: step:6K smpl:384K ep:410 epch:5.12 loss:1.011 grdn:1.017 lr:4.1e-05 updt_s:2.372 data_s:0.074 smp/s:26 mem_gb:25.86 discrete_ce_loss:1.005 discrete_z_loss:0.000 action_flow_loss:0.005

Deployment notes

1. Do not pass norm_tag — stats are baked into the processor files.
2. Task string must match training exactly: Put all oranges in the bowl.
3. Raise the client gripper rate limit: --robot.max_gripper_delta=0.05 (the data contains gripper commands up to 0.05/tick; the 0.03 default throttles grasps).
4. All three cameras at 16:9 (training was 480×270); the docs' 640×480 top camera is a silent train/deploy mismatch.

lerobot-policy-server \
  --policy.pretrained_name_or_path=ASethi04/MolmoAct2-BimanualYAM-oranges-20k \
  --policy.model_dtype=bfloat16 --policy.device=cuda --host=0.0.0.0 --port=8081

Limitations

Single task, 80 demonstrations, no held-out validation set. Validate on hardware with a no-motion action probe before arming the robot.