tau-0-vla

τ₀-VLA Checkpoint

<p align="center"> <a href="https://tau0-vla.github.io/">🌐 <strong>Project Website</strong></a> · <a href="https://arxiv.org/abs/2608.16885">📄 <strong>Paper</strong></a> · <a href="https://github.com/sii-research/tau-0-vla">💻 <strong>Code</strong></a> </p> <p align="center"> <img src="https://raw.githubusercontent.com/sii-research/tau-0-vla/main/assets/overview.png" alt="τ₀-VLA overview" width="100%"> </p>

This repository contains the official pretrained checkpoint for the low-level vision-language-action policy of τ₀-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation. It is intended as an initialization for embodiment- and task-specific post-training.

The complete τ₀-VLA system additionally uses high-level proposal and reflection models, a world model, and a value model for test-time search. Those high-level components are described in the paper and are not part of this checkpoint.

Model releases

Model summary

The low-level policy was trained on 40,115 hours of heterogeneous real-world robot data with multimodal co-training. See the paper for the complete training procedure and system-level evaluation.

Post-training

The reference environment uses Python 3.11, CUDA 12.8, and PyTorch 2.7.1.

git clone https://github.com/sii-research/tau-0-vla.git
cd tau-0-vla
bash scripts/setup.sh

hf download sii-research/tau-0-vla \
  --local-dir checkpoints/tau-0-vla-base

Post-train on the bundled AgiBot World example:

bash scripts/train.sh configs/example_agibot_world_gong/train.yaml \
  --model_name_or_path checkpoints/tau-0-vla-base

The Python loader accepts either a Hub model ID or a local checkpoint path. The quick start downloads the checkpoint first because the training launcher uses offline Hub access by default. The example uses a small Strike the gong subset derived from AgiBot World Beta and stored in LeRobot v3.0 format.

For a new task or embodiment, start from:

• configs/_template/
• src/tau0_vla/adapters/_template/

Input and output contract

The model uses a unified 40-dimensional state/action layout:

Model-facing tensors have the following shapes:

state        (40,)       float32
action       (30, 40)    float32
state_mask   (40,)       float32
action_mask  (40,)       float32

State and action masks are independent. Inactive slots are zero and have mask value 0; changing the mapping, mask, camera order, or normalization changes the action semantics. Arm-joint and end-effector targets are relative to the current state, while gripper, waist, and chassis-velocity targets remain absolute. Adapters restore the output to native robot order after unnormalization.

The public code includes the following stable unified mapping:

Detailed contracts:

• Dataset format
• Data pipeline
• Robot adapters and 40D mapping
• Serving and action restoration

Scope and evaluation

This is a pretrained checkpoint intended to initialize task- and embodiment-specific fine-tuning. For downstream use, fine-tune it on data from the target task and deploy the resulting policy checkpoint. The closed-loop and test-time-computation results in the paper use task-specific fine-tuned policies and, where applicable, the complete hierarchical τ₀-VLA system.

Serving also requires the policy manifest, run specification, Data Spec, and normalization artifacts written by fine-tuning. Run the public server against the resulting fine-tuned policy output.

The bundled AgiBot World subset is a worked post-training example rather than the checkpoint's pretraining corpus.

Intended use and limitations

• Intended uses include robot-learning research, embodiment adaptation, and task-specific post-training.
• A target deployment must provide the correct adapter, camera ordering, 40D mapping, masks, normalization statistics, and native action restoration.
• Hardware serving uses joint-control checkpoints. The LIBERO checkpoint uses a dedicated EEF simulation server.
• The checkpoint is not demonstrated as a zero-shot controller for arbitrary robots or tasks.
• Before physical deployment, validate open-loop behavior and enforce workspace, velocity, collision, human-supervision, and emergency-stop safeguards.

Citation

@misc{cai2026tau0vla,
  title  = {{$\tau_0$-VLA: a Hierarchical Robot Foundation Model with
            World-Model-Guided Test-Time Computation}},
  author = {Xiaowei Cai and Yunuo Cai and Bingao Chen and Jingxiao Chen and
            Zhi Chen and Siyuan Feng and Tengyu Hou and Jingshun Huang and
            Han Jiang and Runkun Ju and Dong Li and Mingxiang Li and
            Shaowei Li and Xinchen Li and Yifan Li and Yi Liu and
            Zhongyuan Liu and Jianlan Luo and Junwen Miao and Ruiqi Ni and
            Buqing Nie and Mingjie Pan and Xinlin Ren and Jianheng Song and
            Jiaxu Wang and Peiqi Wang and Sen Wang and Xiaoyan Wang and
            Dafeng Wei and Dongming Wu and Pengwei Xie and Pu Yang and
            Hangjian Ye and Xiangyu Yue and Jinyu Zhang and Qinglin Zhang and
            Xueyong Zhao and Yue Zhou},
  year   = {2026},
  url    = {https://arxiv.org/abs/2608.16885}
}

License

The model weights and source code are released under the Apache License 2.0. The bundled example data is separately licensed under CC BY-NC-SA 4.0.