HiMoE-VLA-Base

HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies

This repository contains the weights for HiMoE-VLA, a vision-language-action (VLA) policy framework designed to handle heterogeneous robotic datasets with a hierarchical mixture-of-experts.

• Paper: HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies
• Repository: GitHub

Model Description

HiMoE-VLA introduces a Hierarchical Mixture-of-Experts (HiMoE) action module that progressively abstracts away differences across embodiments, action spaces, sensor setups, and control frequencies. It utilizes Action-Space MoE layers at the boundaries for distinct action spaces, Heterogeneity-Balancing MoE layers for residual variation, and dense Transformer blocks in the middle to integrate shared representations.

Sample Usage

For real-world deployment or local inference, you can load and run the policy using the following snippet:

from moevla.policies import policy_config as _policy_config
from moevla.training import config as _config

# Specify these parameters
train_config = ""
dataset_config = ""
checkpoint_dir = ""

policy = _policy_config.create_trained_policy(
    _config.get_training_config(train_config),    
    _config.get_dataset_config(dataset_config), 
    checkpoint_dir, 
    default_prompt=None
)

# Run inference on an example
example = {
    "observation/exterior_image_1_left": ...,
    "observation/wrist_image_left": ...,
    "prompt": "fold clothes"
}

action_chunk = policy.infer(example)["actions"]

Citation

If you find our work useful in your research, please consider citing our paper:

@article{du2025himoe,
  title={HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies},
  author={Du, Zhiying and Liu, Bei and Liang, Yaobo and Shen, Yichao and Cao, Haidong and Zheng, Xiangyu and Feng, Zhiyuan and Wu, Zuxuan and Yang, Jiaolong and Jiang, Yu-Gang},
  journal={arXiv preprint arXiv:2512.05693},
  year={2025}
}