Downloads · 30 days
10
11% of all-time downloads
espejelomar/smolvla-so101-can-butler
smolvla-so101-can-butler is a robotics model from espejelomar. Use it for the robotics task on the model card, and read the license before you ship it in a product. It is set up for lerobot. The card lists the license as apache-2.0.
<img src="media/banner.png" alt="SO-101 arm grasping a can under an overhead depth camera" width="100%"/
Downloads · 30 days
10
11% of all-time downloads
All-time downloads
87
Public
Parameters
450M
911 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors907 MB · 99%
How the weights are stored.
BF16447M · 99%
From the Hugging Face model README
A SmolVLA fine-tune that makes a $100 SO-101 arm hand a can to a person: it waits until a human hand appears on the mat, reaches, grasps the can, and delivers it.
Trained on 26 teleoperated demonstrations (plus 16 "stay still" negatives) from espejelomar/so101-can-butler, for 20k steps on a rented RTX 4090 — about $2.30 of compute.
An autonomous grasp by this checkpoint. Left: the overhead camera. Right: the wrist camera — what the policy sees as the jaws close.
<img src="media/grasp_dualcam.gif" alt="Dual-camera view of an autonomous grasp" width="100%"/>3 grasps out of 3 autonomous attempts, each one verified by the servos rather than by eye (see Verification below):
| attempt | jaw-block gap | verdict |
|---|---|---|
| 1 | 5.91 | held |
| 2 | 4.41 | held |
| 3 | 8.58 | held |
Threshold for "held" is 1.5; failed grasps on this bench measure between −0.41 and 0.52. For honest context: this is a small sample — a larger rate campaign is the next step, and the numbers here should be read as "it works" rather than as a success rate.
A same-data ACT baseline reached 1/3 on the same bench with the same judge; a hand-calibrated classical pipeline (camera → arm homography + scripted grasp) reached 0 autonomous grasps, because absolute camera-to-arm calibration on this setup carried 12–70 mm of error depending on where the can sat. An end-to-end policy needs no such calibration, which is the main practical finding.
These servos are position-controlled: the gripper reports the angle it was told to reach, so gripper position alone cannot distinguish a real grasp from closing on air. What can: a can physically prevents the motor from reaching its commanded angle.
Score = median(achieved − commanded) over the hold. Across 26 human demonstrations with a can genuinely held, this gap never fell below 1.84 (median 7.57). Across 22 failed robot attempts it stayed between −0.41 and 0.52. No false positives were observed. Every result above was scored this way, from the recorded episode, with no human labelling.
The checkpoint expects SmolVLA's own camera names, so map this bench's cameras when running it:
python -m lerobot.scripts.lerobot_rollout \
--policy.path=espejelomar/smolvla-so101-can-butler \
--policy.n_action_steps=50 \
--rename_map='{"observation.images.top": "observation.images.camera1",
"observation.images.wrist": "observation.images.camera2"}' \
--robot.type=so101_follower --robot.port=/dev/ttyACM0 \
--fps=15 --dataset.single_task="Hand me the can"
Notes:
n_action_steps must not exceed the model's chunk_size (50).| base | lerobot/smolvla_base (frozen VLM, action expert trained) |
| data | 42 episodes (26 positive + 16 negative; 2 excluded for early motion) |
| steps / batch | 20,000 / 32 |
| hardware | 1× RTX 4090, ~1h42m, ~$2.30 |
| framework | LeRobot v0.6.1 |