smolVLA

SmolVLA: Optimized for AMD ROCm

SmolVLA (vision-language-action) is a behavior-cloning policy from Hugging Face LeRobot for 6-DOF robot arm control. This repository packages inference for robot arm action prediction using PyTorch, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs.

This is based on the implementation of SmolVLA found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the smolVLA AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.

---

Task Overview

Task: Robot arm action prediction (vision-language-action)

Dataset: BlankHead/so101_redcube_greencloth_3cams (LeRobot format)

Output metrics: MAE, RMSE (per-joint and per-episode)

PyTorch note: CPU runs FP32; GPU runs BF16. No NPU (VitisAI) path is available — make setup-npu, make benchmark-npu, and make evaluate-npu print an informational note and exit cleanly.

---

AMD ROCm Optimization

This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs. Key points:

• Validated backend: PyTorch (native ROCm HIP kernels), FP32 on CPU, BF16 on GPU.
• No code changes required versus the upstream SmolVLA implementation — only environment/runtime configuration differs.
• No NPU (VitisAI) fallback path is available for this model.

---

Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the smolVLA on GitHub.

---

Model Details

Model Type: Vision-language-action policy for robot arm control

Base Model: lerobot/smolvla_base

Model Stats:

• Precision tested: FP32 (CPU), BF16 (GPU)
• Configurable runtime knobs: rtc_config.enabled (Real-Time Chunking), num_steps (flow-matching denoising passes per chunk), n_action_steps (actions consumed per chunk)

---

Accuracy Pipeline

Open-loop offline evaluation is fully implemented: make evaluate-<device> runs inference on recorded dataset episodes and computes per-joint and per-episode MAE / RMSE against the recorded ground-truth actions. Lower is better for both metrics.

Metrics Explained

Accuracy Results

Published Results — Dataset: BlankHead/so101_redcube_greencloth_3cams (13 episodes, chunked_rtc mode):

<!-- accuracy-table-start --> <!-- accuracy-table-end -->

Per-joint breakdown:

---

Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 View the full project on GitHub

The GitHub repository includes:

• Setup and prerequisites for ROCm environments
• Open-loop dataset evaluation with trajectory plots and comparison videos
• Latency benchmarking with Chrome trace output
• PEFT/LoRA-aware checkpoint loading and runtime chunking configuration