gr00t-n16-datagen-v1-stack-joint-2cam-promptnl

GR00T-N1.6 - Stack-Retrieve (joint, 2-cam)

NVIDIA Isaac GR00T-N1.6-3B fine-tuned on the ManiGuard stack base task (sim Franka Panda). Part of the ManiGuard VLA benchmark - GR00T vs pi0.5 on the same task families with identical data, cameras, and controller.

Model

• Base: nvidia/GR00T-N1.6-3B - Eagle (nvidia/Eagle-Block2A-2B-v2) VLM + flow-matching DiT action head
• Embodiment: NEW_EMBODIMENT - Franka Panda, 8-D joint state/action (7 arm joints + 1 gripper)
• Cameras (2): image_left (overview) + wrist (256x256)
• Action: arm = state-relative chunks, gripper = absolute; 16-step horizon; NON_EEF (joint space)
• Tuning: GR00T-N1.6 default - VLM (LLM + visual) frozen, train projector + diffusion action head (no LoRA)

Training

• 8-card config, DeepSpeed ZeRO-2, bf16, global batch 256, 20720 steps (~2 epochs over 2,652,083 frames), cosine LR (peak 2e-4, sqrt-scaled), warmup 0.05
• Data: IDEAS-Lab-Northwestern/datagen-stack-v1-joint-5cam; videos decoded as H.264 for GR00T's torchcodec loader

Usage

Load with Gr00tPolicy from Isaac-GR00T (n1d6), --embodiment-tag NEW_EMBODIMENT. The included processor/ carries the normalization stats + modality config.

WARNING - Convention (must match at eval): joint-space JointController (absolute joint targets, NON_EEF) + 2 cameras (image_left overview + wrist). A mismatched controller or camera set silently feeds an out-of-distribution input.

Paper & Citation

Part of ManiGuard: paper (arXiv:2608.17386) · code · docs

@misc{peng2026maniguard,
  title         = {{MANIGUARD}: A Benchmark and Data Suite for Specification-Grounded
                   Safety Evaluation and Improvement of Robotic Manipulation},
  author        = {Peng, Yiyan and Wang, Philip and Zhan, Simon Sinong and Lyu, Yiqi
                   and Ni, Zhenyang and Yan, Jixin and Wong, Fiorelli and Jiao, Ruochen
                   and Yin, Hang and Cao, Xinyu and Shao, Huajie and Li, Manling
                   and Zhang, Ruohan and Zhu, Qi},
  year          = {2026},
  eprint        = {2608.17386},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2608.17386},
}