cosmos3-aloha-stackbowls-fd-lora

Cosmos 3 Nano — ALOHA stack_bowls Forward-Dynamics LoRA

LoRA finetune of NVIDIA Cosmos3-Nano (forward-dynamics) on the RoboChallenge stack_bowls bimanual ALOHA dataset — a narrow same-robot/task world model for test-before-rig eval.

• Base: nvidia/Cosmos3-Nano (16B two-tower MoT). Reasoner frozen; generator LoRA only.
• Domain: robomind-franka-dual (id 12, 20-D dual-arm [pos3·rot6d6·grip1]×2).
• LoRA: rank 16 / alpha 32 on q,k,v,o_proj_moe_gen (~15.3M params, 0.101%).
• Data: 1,047 eps / 466k transitions, 30 fps, 3 cams (global + 2 wrists) → concat_view, res 720.
• Action: absolute EEF + quat(xyzw) → backward_framewise rot6d deltas; q01/q99 normalized.
• Training: 4000 iters, lr 5e-4, single RTX PRO 6000 96GB, ~1 s/iter, final loss ~0.09.

Checkpoints

iter_XXXXXXXXX/adapter.safetensors — LoRA adapter at iters 1000/2000/3000/4000. Apply on top of nvidia/Cosmos3-Nano (scale = alpha/rank = 2.0).

Eval video (videos/)

ep1046_GT_Pred_stacked.mp4 — complete held-out episode 1046 (496 frames @30fps), teacher-forced forward dynamics. GT (top) / Pred (bottom), labelled. The finetuned generator tracks the real bowl/arm scene faithfully across the full episode.

Reproduce (code/)

Dataset loader, datapacker, experiment config, normalizer, and training TOML.