RoboDual-Specialist

RoboDual

This repo contains checkpoints for the specialist policy of our paper:
Towards Synergistic, Generalized and Efficient Dual-System for Robotic Manipulation

Evaluation with our checkpint

First set your CALVIN_ROOT environment variable wtih:

export CALVIN_ROOT=/path/to/your/calvin_root_path

• Start evaluation on CALVIN (multi-GPU is also supported):

torchrun --standalone --nnodes 1 --nproc-per-node 1 vla-scripts/evaluate_calvin.py \
                                 --generalist_path "/path/to/calvin_generalist" \
                                 --specialist_path "/path/to/calvin_specialist" \
                                 --with_depth \                 # use depth input
                                 --with_gripper \               # use gripper-view inputs (both RGB and depth)
                                 --with_cfg \                   # enable classifier-free guidance
                                 --log_dir calvin

Please refer to vla-scripts/evaluate_calvin.py in our github repo for all evaluation options.

Citation

If you find our code or models useful in your work, please cite our paper:

@article{bu2024robodual,
  title={Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation}, 
  author={Qingwen Bu and Hongyang Li and Li Chen and Jisong Cai and Jia Zeng and Heming Cui and Maoqing Yao and Yu Qiao},
  journal={arXiv preprint arXiv:2410.08001},
  year={2024}
}