groot-bimanual-so100-crlbasket-diffusion-012

GR00T N1.5 — bimanual SO-101, red cube to basket (projector + diffusion action head)

Fine-tune of nvidia/GR00T-N1.5-3B on a self-built 12-DoF bimanual SO-101 rig. CWRU senior capstone (CSDS 395), Fall 2025.

This is one arm of a two-way ablation on tuning scope. Its counterpart, crlbasket-004, freezes the diffusion action head and trains only the projector, and was the variant adopted for the final demo. Full results, evaluation rollouts and project context live on that card.

What differs

Identical dataset, identical 20,000 steps, identical batch size 12. One flag.

The two losses are effectively identical — 0.00003 apart. In informal side-by-side rollouts the frozen-head variant appeared to reach the grasp more consistently, so crlbasket-004 was adopted and received the bulk of the evaluation. That comparison was eyeballed, not scored against a fixed protocol — it is not a measured success-rate difference, and the rollouts are published so anyone can score them properly.

The part that holds regardless: offline loss carried no information about which checkpoint to pick.

Training

• Dataset: bimanual-crlbasket-rblock-merged-00 — 360 episodes / 238k frames / 3 cameras
• 20,000 steps, batch size 12, bf16 via accelerate, single H200
• tune_projector=true, tune_diffusion_model=true, tune_visual=false, tune_llm=false, lora_rank=0

Recorded rollouts: eval_bimanual-crlbasket-diffusion-012 (10). An earlier variant of this configuration was also evaluated at a 10 Hz control rate (eval_…-diffusion-003-002).

Limitations

Same as the counterpart: single task, discretized object positions, occlusion is unrecovered, and success rates are demonstration-grade rather than production-grade. See crlbasket-004.

---

How to Get Started with the Model

For a complete walkthrough, see the training guide. Below is the short version on how to train and run inference/eval:

Train from scratch

lerobot-train \
  --dataset.repo_id=${HF_USER}/<dataset> \
  --policy.type=act \
  --output_dir=outputs/train/<desired_policy_repo_id> \
  --job_name=lerobot_training \
  --policy.device=cuda \
  --policy.repo_id=${HF_USER}/<desired_policy_repo_id>
  --wandb.enable=true

Writes checkpoints to outputs/train/<desired_policy_repo_id>/checkpoints/.

Evaluate the policy/run inference

lerobot-record \
  --robot.type=so100_follower \
  --dataset.repo_id=<hf_user>/eval_<dataset> \
  --policy.path=<hf_user>/<desired_policy_repo_id> \
  --episodes=10

Prefix the dataset repo with eval_ and supply --policy.path pointing to a local or hub checkpoint.

---

Model Details

• License: apache-2.0