gr00t-kinova-red-block-on-plate

GR00T N1.7 fine-tuned on Kinova "put the red block in the pot"

Fine-tuned from the generic base nvidia/GR00T-N1.7-3B on olingoudey/kinova_put_red_block_on_plate — real-world teleop data from the same Kinova arm (tidybot2) as rohansiva/gr00t-kinova-clear-table, single task, 50 episodes.

Per the dataset's own collection notes: red and green blocks start on the left side of a stationary pot, unstacked, within reach, fixed 3rd-person camera framing, constant background/lighting.

New embodiment

Same NEW_EMBODIMENT custom modality config as the clear_table checkpoint (identical schema, same recording pipeline):

• State (15-dim): arm_joint_angles[0:7], arm_pos[7:10], arm_quat[10:14] (x,y,z,w), gripper_pos[14:15] (0-1 normalized)
• Action (7-dim): pos_delta[0:3] (target-minus-observed position), rot_delta[3:6] (axis-angle rotation vector of q_target · q_obs⁻¹), gripper[6:7] (absolute 0-1 target)
• Two cameras: wrist_image, third_person_image, 640x480, 50fps

Training setup

• Base: nvidia/GR00T-N1.7-3B
• Trainable: everything except the frozen VLM backbone (nvidia/Cosmos-Reason2-2B) — 1.62B / 3.14B params (51.5%)
• 8x GPU, DeepSpeed ZeRO-2
• Global batch size: 192
• 15 epochs (2,240 steps — dataset is 28,673 frames, about 1/4 the size of clear_table's)
• Learning rate: 3e-5, warmup ratio 0.05, weight decay 1e-5

Final train loss: 0.144 (from ~1.22 at start).

Caveats

• Single task, 50 short episodes (28,673 frames total) — narrow, data-scarce fine-tune.
• No held-out/open-loop eval has been run against this checkpoint yet.
• Same untested 16-step action horizon (~320ms at 50fps) as the clear_table checkpoint.