smolvla-base-datagen-v1-clutter-joint-2cam

SmolVLA - Clutter (joint, 2-cam)

HuggingFace LeRobot SmolVLA fine-tuned on the ManiGuard clutter base task (sim Franka Panda). Part of the ManiGuard VLA benchmark - SmolVLA vs pi0.5 vs GR00T on the same task families with identical data, cameras, and controller.

Model

• Base: lerobot/smolvla_base - SmolVLM2 vision-language backbone + flow-matching action expert
• Embodiment: Franka Panda, 8-D joint state/action (7 arm joints + 1 gripper), padded to SmolVLA's 32-D
• Cameras (2): observation.images.top (overview = left view) + observation.images.wrist (256x256)
• Action: absolute joint targets (fed straight to a JointController at eval); 50-step chunk
• Tuning: SmolVLA default - vision encoder frozen, train the action expert (no LoRA)

Training

• 8-GPU accelerate DDP, global batch 512, 3522 steps (~2 epochs over 901,520 frames)
• Data: IDEAS-Lab-Northwestern/datagen-clutter-v1-joint-5cam - the 5-cam datagen dataset, prepared to a 2-cam standard-keyed copy (H.264 videos)

Usage

Load with SmolVLAPolicy.from_pretrained("IDEAS-Lab-Northwestern/smolvla-base-datagen-v1-clutter-joint-2cam-yanZ") from LeRobot. The checkpoint carries the normalization stats.

WARNING - Convention (must match at eval): joint-space JointController (absolute joint targets) + 2 cameras (observation.images.top = the left overview, observation.images.wrist). A mismatched controller, camera set, or overview view silently feeds an out-of-distribution input.

Paper & Citation

Part of ManiGuard: paper (arXiv:2608.17386) · code · docs

@misc{peng2026maniguard,
  title         = {{MANIGUARD}: A Benchmark and Data Suite for Specification-Grounded
                   Safety Evaluation and Improvement of Robotic Manipulation},
  author        = {Peng, Yiyan and Wang, Philip and Zhan, Simon Sinong and Lyu, Yiqi
                   and Ni, Zhenyang and Yan, Jixin and Wong, Fiorelli and Jiao, Ruochen
                   and Yin, Hang and Cao, Xinyu and Shao, Huajie and Li, Manling
                   and Zhang, Ruohan and Zhu, Qi},
  year          = {2026},
  eprint        = {2608.17386},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2608.17386},
}

License

The base model lerobot/smolvla_base is released under the Apache License, Version 2.0. ManiGuard's fine-tuning contributions are also released under Apache-2.0.