MolmoAct2-BimanualYAM-oranges-12k-fixed

Correction (2026-08-06): an earlier revision of this card carried a "vision-blind" warning. That was a measurement error (two concurrent eval jobs' log lines were mistaken for the clean-vs-shuffled comparison). The checkpoint's actual camera-shuffle probe is healthy: ratio 1.28–1.49x across checkpoints, matching the best models in this series. This model is cleared for deployment testing.

MolmoAct2-BimanualYAM — "Put all oranges in the bowl" (12k steps + PR#4249 training fixes)

Fine-tune of allenai/MolmoAct2-BimanualYAM on brandonyang/yam-vive-teleop (80 teleop episodes, 74,927 frames, bimanual YAM, 30 fps).

This is the recommended model of the series. It repeats the recipe of ASethi04/MolmoAct2-BimanualYAM-oranges-12k with the training fixes from upstream lerobot PR#4249 (by the MolmoAct2 author) applied:

• flow-matching noise/timesteps/velocity targets kept in fp32 under autocast (previously quantized to bf16)
• distributed-correct weighted-loss normalization (denominator all-reduced across ranks)
• clamped actions at episode boundaries are supervised (previously masked) — matches the original MolmoAct2 objective and teaches the policy to settle at task completion
• input-embedding freeze no longer over-freezes tied output parameters
• llm_residual_dropout=0.1 (original-recipe regularization, previously missing)

Augmentation keeps the affine transform (±5° rotation, 5% translation): an A/B on the real robot showed removing it hurts. Color jitter is narrowed to Ai2's own ranges (saturation 0.8–1.2, hue ±0.02).

Training

Usage notes

1. inference_action_mode="continuous" is already baked into this checkpoint's config — no override needed (unlike the earlier models in this series).
2. Do not pass norm_tag — normalization stats come from the fine-tuning dataset.
3. Task string must match training exactly: Put all oranges in the bowl.
4. Raise the client gripper rate limit: --robot.max_gripper_delta=0.05 (the training data contains gripper commands up to 0.05/tick; the 0.03 default throttles every grasp).
5. All three cameras should run 16:9 (e.g. 640×360), matching training geometry.

lerobot-policy-server \
  --policy.pretrained_name_or_path=ASethi04/MolmoAct2-BimanualYAM-oranges-12k-fixed \
  --policy.model_dtype=bfloat16 --policy.device=cuda --host=0.0.0.0 --port=8081

Limitations

Single task, 80 demonstrations, no held-out validation set. Validate on hardware with a no-motion action probe before arming the robot.