GR00T-N1.5-dexjoco-single-arm-baseline

GR00T-N1.5-dexjoco-single-arm-baseline

GR00T-N1.5 finetuned on the DexJoCo single-arm multitask set. This is the reference (baseline, non-MoE) policy for the DexJoCo arm of the quantizability gate project.

• Benchmark: DexJoCo, single-arm multitask. The single-arm task set is water_plant, hammer_nail, pick_bucket, pinch_tongs, fold_glasses, click_mouse — 6 tasks, 100 episodes each.
• Embodiment: single arm with a 16-DoF dexterous hand. Embodiment tag in the shipped metadata is new_embodiment.
• Observations: two 640x640 RGB views — front, wrist — at 30 fps
• State: arm position (3) + arm rotation (quaternion, 4) + hand joints (16)
• Actions: arm position (3) + arm rotation (3) + hand joints (16), all absolute, action horizon 16
• Finetuning data: dexjoco_lerobot_datasets, LeRobot v3.0 packed layout
• Checkpoint: step 60,000 = 17.53 epochs (configured for 18 epochs; 60k steps came first). Final training loss 0.0069, from 1.084 at step 0.

A caution for anyone porting the gate here

The action space is absolute, not delta. Risk descriptors written for RoboCasa's delta-action space do not transfer: applied unchanged to DexJoCo the computed risk flags fired on 87% and 95% of chunks — noise, not signal. Any gate for this embodiment needs its descriptors and thresholds re-derived from these actions.

There is also a dual-arm DexJoCo baseline at the same recipe which is not published here.

Architecture

GR00T-N1.5 (model_type: gr00t_n1_5, architectures: ["GR00T_N1_5"]), unchanged from the NVIDIA release:

Training recipe

Read directly out of this checkpoint's trainer_state.json and the run's training_args.bin:

How to load and serve

Requires NVIDIA Isaac-GR00T and transformers==4.51.3 (the version this checkpoint was written with).

huggingface-cli download prehj/GR00T-N1.5-dexjoco-single-arm-baseline --local-dir ./ckpt

from gr00t.model.policy import Gr00tPolicy
from gr00t.experiment.data_config import DATA_CONFIG_MAP

data_config = DATA_CONFIG_MAP["<your dexjoco data config>"]
policy = Gr00tPolicy(
    model_path="./ckpt",
    modality_config=data_config.modality_config(),
    modality_transform=data_config.transform(),
    embodiment_tag="new_embodiment",
    device="cuda",
)
action_chunk = policy.get_action(observation)   # 16 steps

Or serve it over the Isaac-GR00T inference service and drive it from the simulator client:

python scripts/inference_service.py --server \
  --model_path ./ckpt --embodiment_tag new_embodiment --denoising_steps 4

experiment_cfg/metadata.json ships in this repo and carries the state/action normalisation statistics and the modality schema — the policy will not produce correct actions without it, so keep the directory layout intact.

Files