smolvla

Model Card for SmolVLA (IB-Robot)

SmolVLA (Small Vision-Language-Action) policy fine-tuned within the IB-Robot framework. Combines a SmolVLM2-500M vision-language backbone with an action expert for robotic manipulation, packaged with RKNN compiled artifacts for Rockchip RK3588 edge deployment.

Repository Structure

• inference_manifest.json — deployment routing (schema v3)
• config.json — LeRobot policy config (type=smolvla)
• model.safetensors — policy torch weights (~865 MB)
• policy_preprocessor.json + policy_postprocessor.json — normalization steps
• HuggingFaceTB/SmolVLM2-500M-Video-Instruct/ — vendored VLM backbone (12 files, ~1.9 GB)
• artifacts/rknn/rknn_rk3588/ — RKNN compiled modules (5 artifacts)
• train_config.json — full training hyperparameters

Deployment Backends

The RKNN deployment runs a 5-stage pipeline: vision_top / vision_wrist (shared vision encoder) -> embedding -> prefill -> action.

Inputs: observation.state [6], observation.current [6], observation.images.top [3,480,640], observation.images.wrist [3,480,640] Output: action [6] (5 joints + gripper)

Source Model

This bundle's policy weights are fine-tuned from the upstream SmolVLA base model:

• Policy base model (HuggingFace): lerobot/smolvla_base
• VLM backbone (HuggingFace): HuggingFaceTB/SmolVLM2-500M-Video-Instruct

The VLM backbone is vendored locally under HuggingFaceTB/SmolVLM2-500M-Video-Instruct/ for offline deployment. The RKNN artifacts were converted from the torch weights. See scripts/train_policy.sh for training and scripts/convert_hmm.sh for conversion procedures.

Citation

@inproceedings{smolvla,
  title     = {SmolVLA: Democratizing Cost-Efficient Vision-Language-Action Models for Robot Manipulation},
  author    = {LeCun, Yann and others},
  booktitle = {HuggingFace},
  year      = {2025}
}
@software{ib_robot,
  title   = {IB-Robot: Intelligence Boom Robot},
  url     = {https://gitcode.com/openeuler/IB_Robot},
  license = {Apache-2.0}
}