rskill-smolvla-so101-eraser_place-bf16

rskill-smolvla-so101-eraser_place-bf16

OpenRAL rSkill — SmolVLA finetuned to place an eraser on a blue square with a real SO-101 follower arm, packaged for openral deploy run.

This package wraps makermods/smolvla_makermods_eraser_place_unblurry_real_2026-07-31_17-35-54 (Apache-2.0) with an rskill.yaml manifest that adds capability checking, license surfacing, the camera-slot aliasing, the joint-units contract, a training-derived starting pose, latency budgets, and local registry integration. It does not copy model weights.

Preview

Frames from episode 0 of the training dataset — the fixed front view the checkpoint conditions on. The eraser starts beside the blue square and ends on top of it.

The arm-mounted wrist view over the same three moments:

What this skill does

Picks up an eraser from a tabletop and places it on a blue square marked on the same surface, on a real SO-101 follower arm. Single-task policy: it was trained on exactly one instruction and has no other behaviour.

Prompt it verbatim. The dataset carries exactly ONE task string — "place the erase on the blue square", upstream typo included. The tokenizer saw only that string; paraphrasing ("place the eraser on the blue square") puts the language conditioning out of distribution.

Quick start

from openral_rskill.loader import rSkill

pkg = rSkill.from_yaml("rskills/rskill-smolvla-so101-eraser_place-bf16/rskill.yaml")
print(pkg.manifest.name, pkg.manifest.version)

# Does this skill fit your arm?
uv run openral rskill check rskills/rskill-smolvla-so101-eraser_place-bf16 \
    --robot robots/so101_follower/robot.yaml

# Real SO-101 deploy (weights are public Apache-2.0). The skill is discovered
# from the in-tree palette by embodiment match; the reasoner picks it from the
# `eraser` / `place` vocabulary in the manifest.
uv run openral deploy run --config scenes/deploy/so101_bench.yaml

# `deploy run` has no --initial-task (that flag is `deploy sim` only), so send
# the goal to the running graph — verbatim:
uv run openral prompt "place the erase on the blue square"

How it works

SmolVLA (~0.45 B: a 16-layer SmolVLM2-500M vision-language backbone with a frozen vision encoder, plus a flow-matching action expert at expert_width_multiplier 0.75, 10 denoising steps). Two 640×480 RGB views and the 6-D joint-position state go in; one 50-step chunk of absolute 6-D joint targets comes out, replayed synchronously before the next inference.

Observation → action contract

Both views are internally letterboxed to 512×512 by the policy's own resize_imgs_with_padding, so the manifest only requires ≥224×224.

Camera aliasing — which view goes where

The runner first re-keys the robot's sensors onto the VLA slots using robots/so101_follower/robot.yaml's vla_feature_key (top → camera1, wrist → camera2); the manifest's image_preprocessing.aliases then rename those slots to this checkpoint's own input-feature keys.

This mapping is load-bearing and was verified numerically (tests/integration/test_smolvla_eraser_place_rskill.py): replaying a real training frame through the manifest wiring reproduces the recorded teleop chunk to a per-joint MAE of 0.15 – 1.46 over all 50 steps (the flow-matching sampler is stochastic, so this varies run to run; the test's ceiling is 5.0). Swapping the two aliases on the same frame degrades it to 1.04 – 44.27.

Joint units — degrees

State and action are on the lerobot SO-ARM scale this repo models as degrees (the normalizer stats span observation.state −102.9 … +101.1 and action −103.8 … +103.4 — radians would be ±3.14). OpenRAL's JointState / Action contract is radians, so the skill runner converts deg↔rad at the policy boundary. The manifest declares action_contract.joint_units: degrees explicitly rather than relying on the stats-magnitude heuristic — a wrong guess here drives the arm into its joint limits.

Home pose

starting_pose is a real home pose recovered from the training data, not an SRDF default and not "wherever episode 0 began". The operator both starts and returns to it — per-joint medians over all 25 episodes' first frames and last frames agree to two decimals:

(lerobot units.) The median is used rather than the mean: wrist_flex has a few long-start outliers reaching 74.46 that drag the mean to 59.02, while the median sits at 56.70 — and the last-frame median independently agrees. shoulder_lift has std 0.09, i.e. a hard mechanical home stop.

Units are mixed, per the OpenRAL joint-channel contract:

• the five arm joints convert degrees → radians;
• the gripper channel is normalised [0, 1], not an angle — SO100FollowerHAL._obs_to_positions divides the lerobot 0–100 reading by 100, and MujocoArmHAL's AFFINE_LOW_HIGH read mode documents the same [0, 1] public surface. So lerobot 1.95 → 0.0195 (jaws essentially closed), not radians(1.95) = 0.034, which would command a slightly open jaw.

Two arm channels are clamped to the SO-101 manifest limits (see below); both clamps are ≤ 5° and stay inside the observed home spread, so the first VLA tick still sees an in-distribution pose.

starting_pose: [-0.1258, -1.7453, 1.5708, 0.9897, 0.1074, 0.0195]

Known limitation — the training envelope overshoots the SO-101 limits

The teleop data reaches slightly past the SO-101 MJCF new_calib limits carried by robots/so101_follower/robot.yaml:

Those are the tails of a per-arm servo calibration, not a different kinematic chain — but the safety kernel clamps or rejects anything past the manifest limits, so expect clipping at the very top of the elbow_flex / wrist_flex range. Do not widen the robot manifest to accommodate this checkpoint: the limits are the mechanical contract, and loosening them is a safety-WG decision with a hazard-log entry (CLAUDE.md §3 "Safety"), not a packaging change.

How it was trained

Weights stay upstream — this rSkill does not mirror them. Every OpenRAL fetch path for this manifest is a per-file hf_hub_download (config.json, model.safetensors, the two processor JSONs and their normalizer .safetensors), never a snapshot_download, so the 20 checkpoints/<step>/ training snapshots in that repo (≈ 21 GB with optimizer state) are never pulled.

Supported robots

Sensors required

Manifest summary

Full schema: openral_core.schemas.RSkillManifest.

Reproduction

just sync --group sim --group dataset --group opencv --inexact

# Replays a real training frame through the manifest's wiring and compares the
# emitted 50-step chunk against the recorded teleop actions.
uv run pytest tests/integration/test_smolvla_eraser_place_rskill.py -q

Evaluation

No benchmarks shipped — there is no simulated twin of this scene, and OpenRAL has not run the checkpoint on hardware. The upstream model card reports no success rates either. The integration test above is a wiring / fidelity check against the training distribution, not a task success rate.

License

This rSkill package (rskill.yaml, README.md, media/) is Apache-2.0. The wrapped weights at hf://makermods/smolvla_makermods_eraser_place_unblurry_real_2026-07-31_17-35-54 are Apache-2.0 as published by the author, as is the training dataset. Commercial use is allowed.

See also

• robots/so101_follower/robot.yaml — RobotDescription manifest.
• scenes/deploy/so101_bench.yaml — paired real-hardware deploy scene.
• rskills/smolvla-so101-pen / rskills/smolvla-so101-pick-place-pen — sibling SO-101 SmolVLA skills.
• docs/reference/vla_compatibility.md — VLA × Robot × Sim matrix.
• CLAUDE.md §3 — rSkill packaging contract.