gr00t-n15-libero-spatial-ppo-bf16

GR00T N1.5 PPO on LIBERO-Spatial — step 10

PPO RL fine-tuning of NVIDIA Isaac GR00T N1.5 on LIBERO-Spatial, pi-RL style, trained with RLinf.

Base model: RLinf/RLinf-Gr00t-SFT-Spatial.

Training progress at this checkpoint

Training-rollout success_once (Flow-SDE exploration noise 0.5, 64 envs x 8 rollout epochs):

critic/explained_variance rises -0.096 -> 0.67 by step 5 and holds; actor/grad_norm ~8.5, stable.

Configuration

model_type: gr00t            # dispatches to the n1d5 implementation
num_action_chunks: 5         # denoising_steps 4, noise_level 0.5
tune_visual: false           # VLM frozen, action head trained
tune_llm: false
update_epoch: 4
clip_ratio_high/low: 0.2     clip_ratio_c: 3.0   value_clip: 0.2
gamma: 0.99                  gae_lambda: 0.95    huber_delta: 10.0
reward/logprob/entropy_type: chunk_level
micro_batch_size: 128        global_batch_size: 1024
lr: 5e-6                     value_lr: 1e-4      weight_decay: 0.01
env: 64 envs x 8 rollout epochs x 240 max steps
seed: 1234                   max_steps: 100      save_interval: 10

Hardware: 4x H100 80GB, ~21 min/optimizer step.

Precision — deviation from the original spec

This checkpoint is bfloat16, not fp32 (Counter({'torch.bfloat16': 907})).

The original plan called for fp32 master weights/optimizer/checkpoints with bf16 compute. That combination is not achievable on GR00T N1.5:

• param_dtype: bf16 -> CategorySpecificLinear.forward does torch.bmm(bf16_activations, fp32_W); its weights are a raw indexed nn.Parameter that FSDP never casts. Raises expected scalar type Float but found BFloat16.
• param_dtype: fp32 -> modeling_eagle2_5_vl.py asserts _attn_implementation == "flash_attention_2" and forces it on the vision tower; FlashAttention rejects fp32. No sdpa/eager path exists in that file.

Satisfying the fp32 requirement would need a cast inside vendored NVIDIA code. Stock bf16 was used instead.

Contents

actor/model_state_dict/full_weights.pt   5.46 GB   portable weights
actor/dcp_checkpoint/                    9.76 GB   4-rank distributed resume state