smolvla_lehome_2

LeHome Challenge 2026 — Policy Evaluation Guide

Policy Overview

• Model: SmolVLA (LeRobot, SmolVLM2-500M-Video-Instruct backbone)
• Task: Bimanual garment folding (all 4 categories: top_long, top_short, pant_long, pant_short)
• Training: 30,000 steps, batch size 32, on merged dataset of 1000 demonstrations (250 per garment type)
• Policy Repository: [TODO: insert HuggingFace repo link or Docker image URI]

---

Setup

Option A: UV (recommended)

git clone https://github.com/lehome-official/lehome-challenge.git

cd lehome-challenge
uv sync

cd third_party
git clone https://github.com/lehome-official/IsaacLab.git
cd ..

source .venv/bin/activate
./third_party/IsaacLab/isaaclab.sh -i none

uv pip install -e ./source/lehome

Option B: Docker

See docs/docker_installation.md.

---

Assets & Dataset

# Simulation assets (scenes, objects, robots)
hf download lehome/asset_challenge --repo-type dataset --local-dir Assets

# Dataset metadata (required for LeRobot policy loading)
hf download lehome/dataset_challenge_merged --repo-type dataset --local-dir Datasets/example

---

Checkpoint

Download the policy checkpoint from the repository linked above and place it at:

outputs/train/smolvla_four_merge_3/checkpoints/

---

Evaluation

Run evaluation for each garment category. The policy uses a unified task description across all garment types.

# Top Long
PYNPUT_BACKEND=dummy xvfb-run -a python -m scripts.eval \
    --policy_type lerobot \
    --policy_path outputs/train/smolvla_four_merge_3/checkpoints/last/pretrained_model \
    --garment_type "top_long" \
    --dataset_root Datasets/example/four_types_merged \
    --num_episodes 5 \
    --enable_cameras \
    --device cpu \
    --headless \
    --task_description "fold the garment on the table"

# Top Short
PYNPUT_BACKEND=dummy xvfb-run -a python -m scripts.eval \
    --policy_type lerobot \
    --policy_path outputs/train/smolvla_four_merge_3/checkpoints/last/pretrained_model \
    --garment_type "top_short" \
    --dataset_root Datasets/example/four_types_merged \
    --num_episodes 5 \
    --enable_cameras \
    --device cpu \
    --headless \
    --task_description "fold the garment on the table"

# Pant Long
PYNPUT_BACKEND=dummy xvfb-run -a python -m scripts.eval \
    --policy_type lerobot \
    --policy_path outputs/train/smolvla_four_merge_3/checkpoints/last/pretrained_model \
    --garment_type "pant_long" \
    --dataset_root Datasets/example/four_types_merged \
    --num_episodes 5 \
    --enable_cameras \
    --device cpu \
    --headless \
    --task_description "fold the garment on the table"

# Pant Short
PYNPUT_BACKEND=dummy xvfb-run -a python -m scripts.eval \
    --policy_type lerobot \
    --policy_path outputs/train/smolvla_four_merge_3/checkpoints/last/pretrained_model \
    --garment_type "pant_short" \
    --dataset_root Datasets/example/four_types_merged \
    --num_episodes 5 \
    --enable_cameras \
    --device cpu \
    --headless \
    --task_description "fold the garment on the table"

Key Parameters

Without Display (Headless Server)

PYNPUT_BACKEND=dummy xvfb-run -a python -m scripts.eval ...

---

Self-Reported Results

See rollout_results.txt for detailed per-garment results.

Issue

Pytorch version not match

solution

uv pip install --upgrade --no-cache-dir --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu128

Complete problem statment

CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA to enable device-side assertions

NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation. The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_61 sm_70 sm_75 sm_80 sm_86 sm_90. If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at Start Locally | PyTorch