rskill-smolvla-aloha_agilex-robotwin-bf16

rskill-smolvla-aloha_agilex-robotwin-bf16

OpenRAL rSkill — SmolVLA (0.45 B) finetuned on the RoboTwin 2.0 unified dual-arm dataset (50 bimanual SAPIEN tasks, aloha-agilex embodiment), packaged for use with the OpenRAL robot agent framework.

This package wraps lerobot/smolvla_robotwin with a rskill.yaml manifest that adds capability checking, license surfacing, latency budgets, and local registry integration. It does not copy model weights.

What this skill does

A multi-task dual-arm policy for the RoboTwin 2.0 benchmark (Chen et al., arXiv 2506.18088). Action chunks of length 50 across three RGB views (head + per-wrist) driving a 14-DoF dual-arm joint command on the AgileX "aloha-agilex" embodiment.

How it works

OpenRAL loads the upstream LeRobot SmolVLA policy from hf://lerobot/smolvla_robotwin and uses the in-tree smolvla adapter to run chunked inference. RoboTwin itself runs in a separate SAPIEN/CuRobo sidecar; the sidecar returns three RGB views plus the 14-D aloha-agilex joint state, and the adapter replays 50-action chunks as absolute 14-D joint position commands.

Sensors / observation contract

Manifest summary

How to run it

RoboTwin runs on SAPIEN out-of-process via a Python 3.10 sidecar — its stack is incompatible with the openral 3.12 venv. Provision the sidecar venv, then:

# openral-side wire (pyzmq + msgpack)
just sync --all-packages --group robotwin --inexact

# single task
openral benchmark scene \
  --config scenes/benchmark/robotwin_lift_pot.yaml \
  --rskill rskills/smolvla-robotwin

# the 5-task suite
openral benchmark run --suite robotwin --vla smolvla:rskills/smolvla-robotwin

The SAPIEN+RoboTwin sidecar provisioning recipe uses OPENRAL_ROBOTWIN_AUTO_PROVISION=1 or the manual conda recipe.

Provenance

• Weights: lerobot/smolvla_robotwin (Apache-2.0), base lerobot/smolvla_base.
• Dataset: lerobot/robotwin_unified (Apache-2.0; pepijn223/robotwin_unified_v3 renamed).
• Eval protocol: RoboTwin official — 100 episodes/task, sim built-in success, episode_length=300. No locally-reproduced official numbers shipped yet (eval/ is empty). The current website artifact is a 150-step GPU openral benchmark scene smoke clip (robotwin_smolvla-robotwin_fail.mp4, success=False) for visual validation only; populate eval/ with openral benchmark run --suite robotwin on the eval host.

STATE NOTE: the live RoboTwin sidecar returns a 14-D aloha-agilex state, and the official policy_preprocessor.json normalization stats expect observation.state shape (14,); rskill.yaml pins state_contract.dim: 14 accordingly (confirmed via live verification).

License

This rSkill wrapper, the upstream lerobot/smolvla_robotwin checkpoint, and the lerobot/robotwin_unified dataset are Apache-2.0. The package does not copy weights into this repository; runtime loading still emits OpenRAL's unverified-provenance warning until the planned signing control exists.