Downloads · 30 days
21
12% of all-time downloads
YieumYoon/groot-bimanual-so100-crlbasket-004
groot-bimanual-so100-crlbasket-004 is a robotics model from YieumYoon. Use it for the robotics task on the model card, and read the license before you ship it in a product. It is set up for lerobot. The card lists the license as apache-2.0.
Fine-tune of nvidia/GR00T-N1.5-3B for a language-conditioned bimanual pick-and-place task on a self-built 12-DoF SO-101 rig. CWRU senior capstone (CSDS 395), Fall 2025.
Downloads · 30 days
21
12% of all-time downloads
All-time downloads
180
Public
Parameters
2.4B
7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors7 GB · 100%
How the weights are stored.
BF161.3B · 56%
From the Hugging Face model README
Fine-tune of nvidia/GR00T-N1.5-3B for a
language-conditioned bimanual pick-and-place task on a self-built 12-DoF SO-101 rig.
CWRU senior capstone (CSDS 395), Fall 2025.
This is the adopted model of a two-way ablation on tuning scope — see Ablation below.
Task: "Grab the red cube and put it in a red basket"
| Task success | |
|---|---|
| Untuned GR00T N1.5 on this embodiment | 0% — could not complete the task at all |
| This model, clean scene | ~25% |
| This model, distractors present | ~10% |
Mean end-to-end inference latency: 211 ms (LeRobot latency benchmark). Physical execution latency is higher, bounded by the hardware.
Recorded rollouts are public — these are the raw evaluation episodes, not summary statistics:
eval_…-004-all-pos-check — 24 rollouts across cube positionseval_…-004-all-pos-obstacle — 23 rollouts with an obstacle in the workspaceeval_…-004-c-l2 / c-r3 — 5 rollouts each at single positionsbimanual-crlbasket-rblock-merged-00 — 360 episodes / 238k frames / 3 camerasaccelerate, single H200 — ~150 mintune_projector=true, tune_diffusion_model=false, tune_visual=false,
tune_llm=false, lora_rank=0Throughput note. Three video streams per sample make decoding, not GPU compute, the bottleneck. The conventional pairing (batch 256 / 16 workers) projected ~15 h; inverting it to batch 12 / 84 workers finished in ~2.5 h on the same GPU. Worth checking before renting a bigger card.
Same dataset, same 20,000 steps, same batch size. The only variable is whether GR00T's diffusion action head is trained. Run twice — once at 180 episodes, once at 360.
| Model | tune_diffusion_model | Avg loss |
|---|---|---|
this (crlbasket-004) | false | 0.00657 |
crlbasket-diffusion-012 | true | 0.00660 |
Average loss differs by 0.00003 — no signal. In side-by-side rollouts the frozen-head variant reached the grasp more consistently, which is the first failure point in this task, so it was adopted and given the bulk of the evaluation.
This comparison was informal. Successes were counted by eye across a small number of rollouts, not scored against a fixed protocol, and the two variants were never run under a matched evaluation. Treat it as an observation that motivated a choice, not as a measured result.
What the run does support is narrower and firmer: the loss curves carried no information about which checkpoint to pick. Whatever separated these two models was invisible offline. If you are fine-tuning GR00T on a small single-task dataset, do not rank checkpoints by loss.
Untested hypothesis for the direction of the effect: with 360 episodes of one task, training the action head erodes the pretrained action prior, while freezing it and adapting only the projector preserves it. Note that the documented default in NVIDIA's and third-party SO-101 tuning guides is to train the action head, so this observation runs against the standard recipe and deserves a proper measurement before anyone relies on it.
robot_type reads bi_so100_follower — that is the LeRobot embodiment class. The hardware is SO-101;
no preset supported this bimanual configuration, so a custom embodiment was registered.Part of a five-person capstone that built a full pipeline: iOS LiDAR/photogrammetry scanning → Meta SAM 3D reconstruction and segmentation → a browser-based 3D goal-state editor → a VLM turning before/after scene views into a natural-language goal → this policy → SO-101 hardware. The rig, the dataset, the training setup and this model were my scope.
For a complete walkthrough, see the training guide. Below is the short version on how to train and run inference/eval:
lerobot-train \
--dataset.repo_id=${HF_USER}/<dataset> \
--policy.type=act \
--output_dir=outputs/train/<desired_policy_repo_id> \
--job_name=lerobot_training \
--policy.device=cuda \
--policy.repo_id=${HF_USER}/<desired_policy_repo_id>
--wandb.enable=true
Writes checkpoints to outputs/train/<desired_policy_repo_id>/checkpoints/.
lerobot-record \
--robot.type=so100_follower \
--dataset.repo_id=<hf_user>/eval_<dataset> \
--policy.path=<hf_user>/<desired_policy_repo_id> \
--episodes=10
Prefix the dataset repo with eval_ and supply --policy.path pointing to a local or hub checkpoint.