rskill-lingbot_va_a1-galaxea_a1-fruit_placement-bf16

rskill-lingbot_va_a1-galaxea_a1-fruit_placement-bf16

OpenRAL rSkill — a LingBot-VA fruit-placement policy for the Galaxea A1, deployed through OpenRAL's observation, typed-action, safety-kernel, and HAL contracts.

This package points to the public checkpoint at pengyue-polaron/lingbot-va-galaxea-a1-fruit-placement-eef and does not copy model weights into the OpenRAL repository.

Preview

What this skill does

The policy picks fruit, including a mango, from a tabletop and places it into a bowl or plate. It consumes synchronized front and wrist RGB observations and predicts episode-relative end-effector pose plus a continuous normalized gripper command.

Upstream model and training

The checkpoint is a full-parameter fine-tune of robbyant/lingbot-va-base. It jointly predicts video latents and robot-action channels. Training used 130 episodes and 44,824 frames at 30 FPS from the revision-pinned nyush-galaxea-a1-fruit-placement-eef-v21 dataset. The run used 1,000 optimizer steps, two NVIDIA H100 80 GB GPUs, full-parameter FSDP, bfloat16, and an effective global batch size of 16.

The model emits 16 EEF/gripper steps per chunk. Quantile normalization and the action-channel map [0, 1, 2, 3, 4, 5, 6, 28] are applied in the external LingBot server from the checkpoint's configs/va_a1_cfg.py; this is not a LeRobot PolicyProcessorPipeline.

The A1 Runtime policy gateway validates each physical EEF target, solves IK, and emits six absolute joint targets plus one normalized gripper target. If an IK solution is farther than the rSkill's feedback-relative joint-step bound, the gateway advances toward that same solution on subsequent 30 Hz ticks and does not consume the next model action until the full solved target can be dispatched. It then writes the dispatched target's FK result into the LingBot KV cache. OpenRAL's thin adapter validates the gateway model and robot contract, then routes the typed proposal through the normal candidate-action, C++ safety kernel, safe-action, and Galaxea A1 HAL path. Runtime's IK is constructed with the active OpenRAL robot manifest's ordered joint limits, so Runtime calibration margins cannot widen the official command envelope.

Sensors and observation contract

The A1 Runtime remains the sole camera-device owner. OpenRAL connects to its versioned paired Camera Bridge and policy gateway over private per-user Unix sockets; it neither imports the Runtime checkout nor opens either RealSense device.

Supported robots

This rSkill is specific to the six-joint Galaxea A1 contract in robots/galaxea_a1/robot.yaml. It is not a generic Cartesian HAL and does not enable the vendor AnyGrasp/AnyEffector path.

Manifest summary

Full schema: openral_core.schemas.RSkillManifest.

Quick start

The software-only compatibility check does not initialize ROS or hardware:

uv run --group lingbot openral rskill check \
  rskills/lingbot-va-galaxea-a1-fruit-placement/rskill.yaml \
  --robot robots/galaxea_a1/robot.yaml

For real deployment, follow the owner-separated startup sequence in docs/methods/01-hal.md: start the A1 Runtime camera owner, LingBot server, and policy gateway; start the isolated OpenRAL ROS1 sidecar; then run the OpenRAL deployment scene. Do not start the A1 Runtime joint execution bridge at the same time.

Evaluation

No formal automatic task-success result is shipped yet. Validation has exercised the real paired cameras, real model server, manifest-to-policy construction, EEF validation and IK, joint-step subdivision, and OpenRAL typed joint/gripper dispatch. Separate real-hardware joint/gripper round trips and a visually verified lemon pick-and-place have traversed candidate action, the C++ safety kernel, safe action, HAL, ROS 1 relay, and the official driver. The non-terminating VLA continued after the visible placement and was stopped by the unchanged joint-solution jump guard; a success detector or bounded episode termination is still needed for a formal task-success result.

License

This rSkill package and the revision-pinned model weights are Apache-2.0. The model repository contains the authoritative LICENSE.txt; the OpenRAL package references the weights and does not redistribute them.

See also

• robots/galaxea_a1/robot.yaml
• scenes/deploy/galaxea_a1_bench.yaml
• docs/methods/01-hal.md