gr00t-kinova-clear-table

GR00T N1.7 fine-tuned on Kinova "clear the table"

Fine-tuned from the generic base nvidia/GR00T-N1.7-3B on olingoudey/kinova_clear_table — real-world teleop data from a Kinova arm (tidybot2), single task ("clear the table"), 50 episodes.

New embodiment

This is a NEW_EMBODIMENT custom modality config (not a pre-registered GR00T embodiment tag):

• State (15-dim): arm_joint_angles[0:7], arm_pos[7:10], arm_quat[10:14] (x,y,z,w), gripper_pos[14:15] (0-1 normalized)
• Action (7-dim): pos_delta[0:3] (target-minus-observed position), rot_delta[3:6] (axis-angle rotation vector of q_target · q_obs⁻¹, confirmed by the data provider), gripper[6:7] (absolute 0-1 target)
• Two cameras: wrist_image, third_person_image, 640x480, 50fps

Training setup

• Base: nvidia/GR00T-N1.7-3B (generic, not LIBERO-specialized — different domain)
• Trainable: everything except the frozen VLM backbone (nvidia/Cosmos-Reason2-2B) — projector I/O layers, VL self-attention refinement, and the diffusion (DiT) action head (1.62B / 3.14B params, 51.5%)
• 8x GPU, DeepSpeed ZeRO-2
• Global batch size: 192
• 15 epochs (8,517 steps)
• Learning rate: 3e-5 (cosine, linear-scaled from the published recipe's 1e-4 @ batch 640)
• Warmup ratio: 0.05, weight decay: 1e-5

Final train loss: 0.0826 (from ~1.24 at start).

Caveats

• Single task, 50 episodes — this is a narrow, data-scarce fine-tune. The low final loss likely reflects fitting these specific 50 demonstrations closely rather than a broadly generalized "clear the table" skill.
• No held-out/open-loop eval has been run against this checkpoint yet.
• Action horizon (16 steps) covers only ~320ms of real motion at this dataset's 50fps control rate — a fairly short planning window that hasn't been revisited from the LIBERO-derived default.