ur5-lawam-best-deployment-20260919-083515

UR5 LaWAM Best Checkpoints: Deployment Bundle

This package contains two complete fine-tuned policies and a portable, offline inference interface. It does not contain a robot hardware driver and never sends motion commands. Both policies completed 5,000 training updates; deployment uses the checkpoint selected by held-out validation flow loss, not the final update.

The baseline is original LaWAM B0, without the custom paired-latent predictor, ActionVAE auxiliary loss, or residual-injection extensions. The full policy keeps the trained custom method. The two configurations and weights are not interchangeable. These offline losses are not robot success rates.

Contents

• Both unchanged best.pt state dictionaries, with complete backbone weights.
• Qwen3-VL tokenizer/processor and backbone architecture configurations.
• DINOv3 and LAM architecture configurations.
• Full-method latent-coordinate metadata and current-state normalization.
• The UR5 train-only action normalization and frame-conversion contract.
• Necessary local policy source, inference API, HTTP server, client, and a sample.
• Dependency versions, license notices, and verification metadata.

Backbones are constructed from configurations and then loaded strictly from the selected complete checkpoint. No separate Qwen, DINO, LAM, ActionVAE checkpoint, training dataset, teacher HDF5 cache, or original training-machine path is needed at runtime. No token is required to run a downloaded copy of this public bundle.

Install

Download the public repository first (no access token is required):

python -m pip install huggingface_hub
hf download Jason2Jason/ur5-lawam-best-deployment-20260919-083515 --local-dir ./ur5-lawam-deploy
cd ur5-lawam-deploy

Tested with Python 3.10, PyTorch 2.6.0 + CUDA 12.4, Transformers 5.2.0 and an H100-class GPU. Use a CUDA GPU with at least 16 GiB available memory for one policy; actual requirements depend on input sizes and runtime. At least 24 GiB host RAM and approximately 15 GiB disk space are recommended. Dependencies must be installed before disconnected/offline use.

python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu124
python -m pip install -r requirements.txt

The portable loader uses PyTorch SDPA; a separately compiled FlashAttention package is not required. It preserves checkpoint dtypes and does not quantize or retrain the weights.

Offline Prediction

From the repository directory:

python predict.py --variant baseline --output prediction.json
python predict.py --variant full --output prediction.json

Both commands use the bundled pair of images and recorded observation. Substitute your own RGB images and observation JSON using --base-image, --wrist-image, and --observation. --steps controls inference denoising steps; default 10.

The Python interface is:

from deployment import UR5Policy

policy = UR5Policy(variant="full", device="cuda:0")
result = policy.predict(
    "base.jpg", "wrist.jpg",
    tcp_pose=[x, y, z, rx, ry, rz],
    gripper_width_mm=measured_width_mm,
    state_frame="robot_base",
    language="pick up the object and place it down",
)
actions = result["actions_robot_base"]  # [8, 7], NOT sent to hardware

The full policy requires the current measured TCP pose and gripper width. TCP position is in metres and orientation is a rotation vector in radians, not Euler angles. state_frame must explicitly match robot_base or the dataset's data frame. The baseline does not condition on proprioception and may omit those state inputs. Each camera image is resized to 256x256 as in fine-tuning.

HTTP Policy Service

python serve.py --variant baseline --host 127.0.0.1 --port 8000
python client.py --url http://127.0.0.1:8000/predict

GET /health reports the loaded variant. POST /predict accepts JSON with base_image_b64, wrist_image_b64, and the same state/instruction fields as the Python interface. The example client encodes the sample images. GPU requests are serialized. The server defaults to loopback and has no built-in authentication; use an authenticated gateway before exposing it beyond a trusted local process.

Action Contract And Hardware Integration

• Output is eight 7D actions: [dx,dy,dz,drx,dry,drz,gripper].
• Nominal control rate is 20 Hz; one chunk spans 0.4 seconds.
• Translation is a displacement in metres, not velocity.
• Rotation increments are world-frame relative rotation vectors. Compose them with orientations as rotations, not by adding Euler angles.
• actions_data uses the training F_data frame. actions_robot_base rotates both translation and rotation increments using the recorded -135-degree yaw transform. Verify that this calibration matches the deployed robot setup.
• The final component is a recorded 0/1 gripper label. gripper_labels applies a 0.5 threshold to the denormalized prediction; it is NOT a measured opening or a controller-specific command. Confirm open/close polarity with the collector.
• UR5 action normalization is provided in configs/action_normalization.json. Never substitute LIBERO statistics or generic simulator action scaling.
• Targets were aligned as observation t to recorded action t+1. They are measured adjacent-pose increments, not verified low-level controller commands.

The first prediction includes CUDA startup overhead. Check warm latency on the deployment hardware. Do not assume inference meets a 20 Hz control deadline: the policy predicts chunks, and a validated asynchronous/controller integration is still needed if generation is slower than chunk execution.

No robot IP, camera serial number, safety limits, RTDE connection, or motion execution code is shipped. Before commanding hardware, validate camera ordering, image/state synchronization, coordinate transforms, inverse normalization, gripper polarity, workspace bounds, velocity/acceleration limits, stale-output handling, and emergency-stop behavior. Start with offline replay and supervised, low-speed tests. Do not directly issue unchecked model outputs as motion commands.

Data And Scope

Fine-tuning used 35 valid episodes from penguin_cyx/2026916_ur5_vla_dataset. Episodes 0-34 were valid; corrupted episode 35 and empty episodes 36-39 were excluded. Training used 30 episodes and validation used [2,23,27,28,30]. Action statistics were fitted on training episodes only. Validation checkpoint selection used 128 sampled windows, not a robot rollout test.

The demonstrations show a narrow tabletop orange-to-tray pick-and-place task, with one repeated language instruction. Generalization to new tasks, objects, camera placements, robot calibrations or environments is not established.

Upstream Components

• LaWAM
• Qwen3-VL-2B-Instruct
• DINOv3 ViT-B/16

See NOTICE.md and LICENSES/ before redistribution or use. The presence of a public download does not remove upstream license restrictions.