act-tshirt-folding-lerobot

act-tshirt-folding-lerobot

ACT (Action Chunking Transformer) policy for bimanual t-shirt folding on the Trossen AI stationary (WidowX AI) platform. Trained with lerobot. This Run Record documents the step-100000 (final) checkpoint.

Demo — on-robot eval

<video controls src="https://huggingface.co/TrossenRoboticsCommunity/act-tshirt-folding-lerobot/resolve/main/media/eval_demo.mp4"></video>

Prompt: Fold the tshirt

Offline on-robot evaluation of this checkpoint (step-100000) on the Trossen AI stationary bimanual platform — t-shirt folding.

TL;DR

• Policy: ACT (Action Chunking Transformer) — ResNet18 vision backbone + transformer encoder/decoder with CVAE.
• Framework: lerobot-latest (--policy.type=act).
• Method: Full fine-tune (no LoRA/adapters) from scratch on the target dataset.
• Robot: Trossen AI stationary bimanual, 14-DoF, 3 cameras, 30 fps.
• Checkpoint: step 100000.

Setup / rig configuration

Recorded on the Trossen AI Mobile kit with the mobile base disabled — arms mounted in a fixed, stationary configuration (functionally equivalent to the stationary kit for this task).

Model configuration (config.json)

Training hyperparameters

Inputs (reproducibility manifest)

• Dataset: TrossenRoboticsCommunity/mobile-baseoff-tshirt-folding-lerobot-v1.
• Framework: lerobot-latest (--policy.type=act).
• Batch size: 8.
• Total steps: 100,000 (this is the final checkpoint).
• Training env / hardware: local RTX 5090 (TOTL workstation, Trossen cloud pipeline).

Outputs

• Weights: model.safetensors (full fine-tune, no separate adapter).
• Config: config.json, train_config.json.
• Processors: policy_preprocessor.json (+ normalizer state), policy_postprocessor.json (+ unnormalizer state).

Evaluation (offline policy eval)

Verified to load and run directly via lerobot-record with --policy.path (no async policy server required for ACT). Run from the lerobot_trossen workspace:

uv run lerobot-record \
    --robot.type=bi_widowxai_follower_robot \
    --robot.left_arm_ip_address=192.168.1.4 \
    --robot.right_arm_ip_address=192.168.1.5 \
    --robot.id=bimanual_follower \
    --robot.cameras='{
        cam_high: {type: intelrealsense, serial_number_or_name: "230322276010", width: 640, height: 480, fps: 30},
        cam_left_wrist: {type: intelrealsense, serial_number_or_name: "230322271348", width: 640, height: 480, fps: 30},
        cam_right_wrist: {type: intelrealsense, serial_number_or_name: "128422270115", width: 640, height: 480, fps: 30}
        }' \
    --display_data=false \
    --dataset.repo_id=hamza-trossen/eval_act_tshirt_folding \
    --dataset.push_to_hub=false \
    --dataset.num_episodes=20 \
    --dataset.episode_time_s=105 \
    --dataset.reset_time_s=25 \
    --dataset.single_task="Fold the tshirt" \
    --policy.path=TrossenRoboticsCommunity/act-tshirt-folding-lerobot \
    --robot.min_time_to_move_multiplier=8.0

The task prompt used during recording and eval was "Fold the tshirt".

Eval Results

Links

• Dataset: https://huggingface.co/datasets/TrossenRoboticsCommunity/mobile-baseoff-tshirt-folding-lerobot-v1