HOI-JEPA

HOI-JEPA official experiment checkpoints

This repository contains the official checkpoints accompanying Not All Action Signals Are Equal: Calibrated Action Conditioning for Interaction World Models.

HOI-JEPA predicts interaction-state latents on top of frozen V-JEPA 2 ViT-L features. The release covers categorical EgoDex action labels and continuous DROID robot commands under the same four action-path settings:

• context: no action conditioning;
• full: always-on action conditioning;
• drop25: complete action-sequence dropout on 25% of training examples;
• gated25: learned action-value gate plus 25% sequence dropout.

Every setting includes seeds 42, 43, and 44. Each seed directory contains model.pt, config.yaml, manifest.json, eval.json, and eval_m3_endpoint.json.

Results represented by these checkpoints

EgoDex uses its official 334,991/3,243 train/test split. DROID does not define an official test split for this forecasting task; DROID-held-out-v1 is the fixed project split with 86,072 training and 9,564 held-out episodes.

Loading

import torch

checkpoint = torch.load("model.pt", map_location="cpu", weights_only=False)
print(checkpoint.keys())

Instantiate the model with the adjacent config.yaml before loading its state dictionary. Training and evaluation code, exact downstream protocols, and compiled CVPR/ECCV manuscripts are available in the GitHub repository.

Scope and licensing

These are research checkpoints, not a closed-loop robot policy. DROID ranking is an offline goal-consistency evaluation, and the DROID split is project-defined. Raw EgoDex, DROID, HOI4D, and V-JEPA 2 assets are not redistributed. Use remains subject to the licenses and terms of the upstream datasets and frozen encoder.