Downloads · 30 days
8
12% of all-time downloads
RahulR3174/smolvla-6dof-pick-place-20k
smolvla-6dof-pick-place-20k is a robotics model from RahulR3174. Use it for the robotics task on the model card, and read the license before you ship it in a product. It is set up for lerobot.
A fine-tuned SmolVLA policy for visual pick-and-place manipulation using a simulated 6-DoF reference arm.
Downloads · 30 days
8
12% of all-time downloads
All-time downloads
67
Public
Parameters
450M
907 MB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors907 MB · 100%
How the weights are stored.
BF16447M · 99%
From the Hugging Face model README
A fine-tuned SmolVLA policy for visual pick-and-place manipulation using a simulated 6-DoF reference arm.
The policy was trained for:
Pick up the red cube and place it on the blue target.
HuggingFaceTB/SmolVLM2-500M-Video-InstructThe policy expects four inputs.
observation.images.camera1
Shape: (3, 384, 384)
observation.images.camera2
Shape: (3, 384, 384)
Both images should be RGB and represented as floating-point tensors in the range:
[0, 1]
observation.state
Shape: (6,)
The six values are:
[joint_1, joint_2, joint_3, joint_4, joint_5, gripper]
A natural-language instruction is provided as:
task
Example:
Pick up the red cube and place it on the blue target.
The policy produces a 6-D logical action:
[joint_1, joint_2, joint_3, joint_4, joint_5, gripper]
The policy is action-chunked with a configured chunk size of 50.
The action should be passed through the checkpoint's included postprocessing pipeline before being sent to the robot/simulator.
After downloading this repository, load the policy from the model directory:
from pathlib import Path
import torch
from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy
model_path = Path("./smolvla-6dof-pick-place-20k")
policy = SmolVLAPolicy.from_pretrained(model_path).eval()
device = "cuda" if torch.cuda.is_available() else (
"mps" if torch.backends.mps.is_available() else "cpu"
)
policy.to(device)
policy.reset()
observation = {
"observation.images.top": top_image,
"observation.images.wrist": wrist_image,
"observation.state": robot_state,
"task": [task_instruction],
}
action = policy.select_action(observation)
print(action)
Use the policy_preprocessor.json and policy_postprocessor.json included in this repository as part of the normal LeRobot policy inference pipeline. The saved processor files contain the preprocessing, normalization, device, tokenization, and action postprocessing configuration used with this checkpoint.
The policy was trained on a two-camera 20 Hz pick-and-place dataset using a reference 6-DoF arm.
The trained visual setup uses:
The model was evaluated in the Genesis simulator with randomized cube and target positions within the training workspace.
The model shows strong performance on the trained red cube → blue target visual configuration and remains robust to several changes in wording of the task instruction.
In experiments without additional training, changing the physical object/target colors (for example, green/yellow or orange/purple) caused a substantial drop in success rate.
Therefore, the checkpoint should be treated as a task-specific research model rather than a color-invariant general-purpose manipulation policy.
The final checkpoint was trained with:
The repository contains the final inference checkpoint.
Training-state files used for resuming training are not included in this inference model package.
The model repository contains:
model.safetensors — trained model weightsconfig.json — policy/model configurationpolicy_preprocessor.json — input preprocessing configurationpolicy_preprocessor_step_5_normalizer_processor.safetensors — saved normalization statepolicy_postprocessor.json — output postprocessing configurationpolicy_postprocessor_step_0_unnormalizer_processor.safetensors — saved action unnormalization statetrain_config.json — training configuration metadatademo/smolvla_successful_pick_place_demo_combined.mp4 — synchronized top + wrist camera demonstrationThe complete Genesis environment, data-collection code, training scripts, evaluation scripts, and deployment code are maintained separately in the project GitHub repository.
This Hugging Face repository is intended to contain the model and model-specific usage information, not the complete project source code.