TokenHSI

TokenHSI Checkpoint Instruction

[2025-04-07] All models have been released!

Please download the checkpoints and organize them as the following file structure:

|-- tokenhsi
|-- lpanlib
|-- body_models
    |-- smpl
        |-- SMPL_FEMALE.npz
        |-- SMPL_MALE.npz
        |-- SMPL_NEUTRAL.pkl
        |-- ...
|-- output
    |-- single_task
        |-- ckpt_traj.pth
        |-- ckpt_sit.pth
        |-- ckpt_climb.pth
        |-- ckpt_carry.pth 
    |-- tokenhsi
        |-- ckpt_stage1.pth
        |-- ckpt_stage2_comp_traj_carry
        |-- ckpt_stage2_comp_sit_carry
        |-- ckpt_stage2_comp_climb_carry
        |-- ckpt_stage2_objShape_chair.pth
        |-- ckpt_stage2_objShape_table.pth
        |-- ckpt_stage2_terrainShape_traj.pth
        |-- ckpt_stage2_terrainShape_carry.pth
        |-- ckpt_stage2_longterm.pth

• We first provide four single-task environments, including path-following, sitting, climbing, and carrying. For each task, we train a specialized policy using AMP. The trained models are:

  • single_task/ckpt_traj.pth
  • single_task/ckpt_sit.pth
  • single_task/ckpt_climb.pth
  • single_task/ckpt_carry.pth

• We then provide our multi-task environment for training the versatile, unified transformer policy. The trained model is:

  • tokenhsi/ckpt_stage1.pth

• Finally, we perform policy adaptation to generalize the learned skills to novel tasks. We list the trained models across various downstream tasks.

  • Skill Composition
    • Follow + Carry: tokenhsi/ckpt_stage2_comp_traj_carry.pth
    • Sit + Carry: tokenhsi/ckpt_stage2_comp_sit_carry.pth
    • Climb + Carry: tokenhsi/ckpt_stage2_comp_climb_carry.pth
  • Object Shape Variation
    • Chair: tokenhsi/ckpt_stage2_objShape_chair.pth
    • Table: tokenhsi/ckpt_stage2_objShape_table.pth
  • Terrain Shape Variation
    • Follow: tokenhsi/ckpt_stage2_terrainShape_traj.pth
    • Carry: tokenhsi/ckpt_stage2_terrainShape_carry.pth
  • Long-horizon Task Completion
    • tokenhsi/ckpt_stage2_longterm.pth

GitHub repo: https://github.com/liangpan99/TokenHSI
arXiv paper: https://huggingface.co/papers/2503.19901
Project page: https://liangpan99.github.io/TokenHSI/