See2Act

See2Act checkpoints

Policies from Learning to See While Learning to Act: Diffusion Models for Active Perception in Robot Imitation (Kuancheng Wang, Vaibhav Saxena, Shuo Cheng, Yotto Koga, Danfei Xu; arXiv:2606.23625).

See2Act is a diffusion model that refines where to look together with what to do: every denoising step moves the camera to a pose computed from the current action estimate, renders the scene from there and conditions the next step on that view. These checkpoints were trained on the four occluded Ravens tasks of the paper with the code and configuration at github.com/KuanchengWang/see2act (configs/see2act.json, 100 scripted demonstrations per task from the See2Act dataset).

Each .pth file holds the network weights and the full configuration (architecture, camera schedule, inference protocol), so it loads without further flags.

Usage

git clone https://github.com/KuanchengWang/see2act.git && cd see2act   # install per the README
hf download harrywang01/See2Act --local-dir checkpoints
python scripts/evaluate.py --checkpoint checkpoints/see2act_bin-picking.pth --n_episodes 50
python scripts/visualize_episode.py --checkpoint checkpoints/see2act_bin-search.pth --seed 200001 --out viz/views.png --gif viz/views.gif

Evaluation episodes are seeded (200001, 200003, ...) and the simulator renders with PyBullet's EGL plugin; the policies expect that renderer.

Integrity

afeb2d49c835b70955c43fb14f3f4d17  see2act_bin-picking.pth
30c41d982f7fa602b8aae1c71f37ba41  see2act_bin-search.pth
569f5edc396c19761cab6f9ea58a42a7  see2act_place-red-in-green.pth
dcfbdb509c55dbe873579b75160076cd  see2act_put-within-shelf.pth

Citation

@misc{wang2026learninglearningactdiffusion,
      title={Learning to See While Learning to Act: Diffusion Models for Active Perception in Robot Imitation},
      author={Kuancheng Wang and Vaibhav Saxena and Shuo Cheng and Yotto Koga and Danfei Xu},
      year={2026},
      eprint={2606.23625},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2606.23625},
}