MolmoAct2-BimanualYAM-oranges-12k-noaffine

MolmoAct2-BimanualYAM — "Put all oranges in the bowl" (12k steps, no geometric augmentation)

Fine-tune of allenai/MolmoAct2-BimanualYAM on brandonyang/yam-vive-teleop (80 teleop episodes, 74,927 frames, bimanual YAM, 30 fps).

This is an ablation of ASethi04/MolmoAct2-BimanualYAM-oranges-12k: identical in every respect except that the RandomAffine augmentation (±5° rotation, 5% translation) is removed.

Why remove it

The cameras are fixed and the action targets are absolute joint poses. A ±5° rotation plus 5% translation shifts the scene by ~24 px on a 480-px-wide image while the action label is unchanged — injected label noise in the visuomotor mapping. Both openpi (no image augmentation at all) and OpenVLA (random crop only) avoid this class of transform. Note the lower training loss here is partly mechanical (a less distorted input is an easier objective) and does not by itself demonstrate better generalization.

Augmentation (3 of 5 sampled per image)

brightness 0.8–1.2 · contrast 0.8–1.2 · saturation 0.8–1.2 · hue ±0.02 · sharpness 0.5–1.5

Training

Final training step: step:12K smpl:768K ep:820 epch:10.25 loss:0.607 grdn:2.165 lr:5.0e-06 updt_s:2.409 data_s:0.074 smp/s:26 mem_gb:25.86 discrete_ce_loss:0.603 discrete_z_loss:0.001 action_flow_loss:0.003

Usage notes

1. Set inference_action_mode="continuous" — the saved config has None.
2. Do not pass norm_tag — stats come from the fine-tuning dataset.
3. Task string must match training exactly: Put all oranges in the bowl.
4. Raise the client gripper rate limit to --robot.max_gripper_delta=0.05; the training data contains gripper commands up to 0.05/tick and the 0.03 default throttles grasps.

Limitations

Single task, 80 demonstrations, no held-out validation set. Validate on hardware with a no-motion action probe before arming the robot.