smolvla-so101-pick-cube

Pick-Cube SO-101 — SmolVLA policy

A SmolVLA policy finetuned to "grab the cube and put it in the box" on the SO-101 arm, from three camera views. Trained on the dobri420/pick-cube-so101 teleoperation dataset.

89% grab rate (72/81) on the real arm — the best of a coverage-vs-optimization sweep, and balanced across reach depth (25 / 20 / 15 cm = 89 / 93 / 85 %).

Training corpus vs. performance

Same workspace half-disk in both panels (rings = reach in cm from the shoulder-pan axis, radials = target angle), so coverage and success line up cell-for-cell. Left: where the training demos actually grasped (SO-101 forward kinematics on the jaw center). Right: where the resulting policy succeeds, measured on a held-out grid of 81 targets (27 per depth).

<table> <tr> <td align="center"><b>training coverage</b> (grasp position)</td> <td align="center"><b>eval</b> (grab rate)</td> </tr> <tr> <td><img src="https://huggingface.co/dobri420/pick-cube-smolvla-so101/resolve/main/grasp-coverage.png" alt="training grasp coverage"/></td> <td><img src="https://huggingface.co/dobri420/pick-cube-smolvla-so101/resolve/main/eval-success.png" alt="eval grab-rate heatmap"/></td> </tr> </table>

Training

• Base: lerobot/smolvla_base (expert finetuned, VLM frozen).
• Data: the first 340 episodes of dobri420/pick-cube-so101 (the five tranches through twist-CCW; the dataset has since grown to 440 with a twist-CW tranche not used here).
• Schedule: 2.56M samples @ batch 64 (40k steps), bf16, on a single B200.
• Inputs: three 256×256 camera views (camera1/2/3) + 6-DoF joint state. Output: 6-DoF action, 50-step chunks.

Usage

from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy

policy = SmolVLAPolicy.from_pretrained("dobri420/pick-cube-so101")

The repo ships the runnable inference bundle: model weights, config.json, and the normalization pre/post-processor pipelines (the mean/std that map real joint units ↔ the model's normalized space live in the *_processor.safetensors — they are required for inference). train_config.json records the full training recipe.

Built with LeRobot.