gr00t_n17_so101_mixed_nvfp4_accuracy_thor

GR00T N1.7 SO-101 mixed-NVFP4 release for Jetson Thor

This bundle pairs the original LeRobot checkpoint with TensorRT engines built for Jetson AGX Thor. It is a mixed-precision runtime artifact, not a universal NVFP4 PyTorch checkpoint.

The engine bundle is tied to the TensorRT/CUDA/L4T stack on which it was built. Rebuild it when the JetPack/L4T, TensorRT, CUDA, model architecture, image grid, batch size, or action horizon changes.

Model description

This repository is an independently packaged, accuracy-qualified deployment derivative of raymondlo84-nvidia/gr00t_n17_baseline_green_2_thor, which was fine-tuned from nvidia/GR00T-N1.7-3B. The policy targets an SO-101 follower arm with one front camera and produces a 16-step, 6-joint action chunk for the task "Pick up the cube and place it in the box."

This release does not claim authorship of GR00T, Cosmos, the source fine-tune, Isaac-GR00T, or LeRobot. The changes in this repository are the offline format conversion, mixed-precision TensorRT export, Thor runtime integration, guarded SO-101 launch path, and validation evidence. Exact source revisions and hashes are recorded in SOURCE_REVISIONS.txt.

Intended use

• Reproduce and study the validated mixed-precision inference path on NVIDIA Jetson AGX Thor.
• Perform motor-disconnected validation before any robot rollout.
• Evaluate the specific SO-101 task and embodiment represented by the source fine-tune.

Out-of-scope use

• Safety-critical, medical, industrial, or unsupervised autonomous operation.
• Deployment on a different robot, camera layout, task, TensorRT/CUDA version, or action horizon without rebuilding and revalidating.
• Treating offline numerical similarity as proof of equal physical task success.

Precision recipe

Validation results

• Hardware: NVIDIA Jetson AGX Thor, SM 11.0, L4T 39.2.0, MAXN with dynamic clocks
• Engine calibration: 32 stratified frames across all 7 SO-101 episodes
• CUDA container: 13.0
• PyTorch: 2.10.0
• ModelOpt: 0.39.0

Numerical parity against PyTorch BF16

Latency and size

Motor-disconnected LeRobot validation passed with finite output shape (1, 16, 6). The measured steady preprocessing median was 8.45 ms and the steady TensorRT policy median was 31.45 ms, for a combined 39.90 ms (25.1 Hz).

These numerical checks are not a robot-task success rate. No motor-connected rollout was run during packaging. Start with a clear workspace and an accessible power/e-stop control.

The promoted accuracy result is evidence/bf16_vs_nvfp4_accuracy_calib32_final.json. The similarly named evidence/bf16_vs_nvfp4_calibration_excluded_multiseed.json records an earlier rejected candidate and is retained for auditability; it is not the engine bundle shipped in engines/.

Layout

checkpoint/   Original LeRobot checkpoint and serialized processors
base_model/   Isaac-format model used to instantiate the LeRobot policy offline
engines/      Promoted 32-sample accuracy-preserving TensorRT engine bundle
evidence/     Accuracy, latency, offline validation, and golden outputs
runtime/      Docker recipe, source patches, validator, and guarded launchers
LICENSE       NVIDIA license inherited by the GR00T-derived weights
LICENSE-NVIDIA-OPEN-MODEL.txt
LICENSE-APACHE-2.0.txt
NOTICE
checksums.sha256
SOURCE_REVISIONS.txt

checkpoint/ and base_model/ intentionally contain different serialized layouts. They are hard-linked to the project inputs on this device to avoid using additional disk blocks. Upload tools materialize both files.

Build the deployment image

From the project folder:

PROJECT_DIR=/home/jetson/groot_n17_so101_nvfp4 \
  ./artifacts/so101_nvfp4_accuracy_release/runtime/build_image.sh

The image is named groot-so101-nvfp4:latest. The build embeds the pinned, locally patched Isaac-GR00T and LeRobot sources, but not model weights, engines, datasets, Hugging Face credentials, or cache files.

Validate with motors disconnected

This command mounts no /dev entries and disables container networking:

HF_CACHE_DIR=/home/jetson/groot_n17_so101_nvfp4/cache/huggingface \
DATASET_DIR=/home/jetson/groot_n17_so101_nvfp4/data/calibration \
  ./artifacts/so101_nvfp4_accuracy_release/runtime/validate_offline.sh

It writes JSON and NPZ evidence under artifacts/so101_nvfp4_accuracy_release/evidence/.

First SO-101 rollout

Review runtime/run_so101_nvfp4.sh, verify the follower port and camera, keep the arm clear, and keep a stop control within reach. The launcher requires an explicit motion acknowledgment:

HF_CACHE_DIR=/home/jetson/groot_n17_so101_nvfp4/cache/huggingface \
FOLLOWER_PORT=/dev/ttyACM0 \
CAMERA_DEVICE=/dev/video0 \
ENABLE_ROBOT_MOTION=YES \
  ./artifacts/so101_nvfp4_accuracy_release/runtime/run_so101_nvfp4.sh

Defaults follow the deployment guide: 10 Hz, 60 seconds, RTC backend with RTC overlap disabled, execution horizon 16, queue threshold 0, and policy action horizon 16. The launcher limits relative joint target changes to 5 degrees by default (MAX_RELATIVE_TARGET) and disables automatic return-to-start motion on shutdown. Stop with Ctrl-C.

Limitations

• Validation observations exclude the 32 engine-calibration frames but come from the same seven recorded episodes used by the project.
• Offline action parity and demonstration imitation error do not establish equal physical rollout success. The required final comparison is a paired, randomized BF16-versus-mixed-precision robot evaluation with a pre-declared non-inferiority margin.
• The safety-clamped comparison assumes perfect tracking and does not model motor lag, camera shift, mechanical variation, or sensor noise.
• TensorRT engines are not portable across arbitrary GPU architectures or software stacks.

Licensing and attribution

This is a mixed-license distribution. No single permissive license covers every file:

The root LICENSE currently limits the GR00T work and derivatives to non-commercial research or evaluation. The GR00T model card and Isaac-GR00T README separately refer to the NVIDIA Open Model License; this repository does not attempt to resolve that upstream inconsistency or grant broader rights. Users must review all included terms and the upstream repositories before use or redistribution.

Licensed by NVIDIA Corporation under the NVIDIA Open Model License.

Built on NVIDIA Cosmos.

See NOTICE for attribution and SOURCE_REVISIONS.txt for pinned provenance. NVIDIA, GR00T, Cosmos, LeRobot, Raymond Lo, and other upstream authors do not necessarily endorse this derivative release.

Publishing to Hugging Face

Authenticate outside this directory and upload the complete release folder. Do not copy a token into the bundle. Preserve the checksums, licenses, notice, engine metadata, source revisions, runtime patches, and this README so consumers can audit the hardware/runtime contract.