NeuralActuator

NeuralActuator pretrained checkpoints

Pretrained checkpoints for the RSS 2026 paper NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception. The model is a Flax Transformer encoder (4 layers, hidden dim 192, 4 attention heads, ~1.4M parameters) over an 8-step history of commanded positions, proprioception and actuator telemetry, with three output heads: joint torque, 3-axis end-effector external force with a contact gate, and per-motor condition logits. The torque head is trained without torque labels by backpropagating a position loss through a differentiable simulator (MuJoCo MJX); the force and condition heads use direct supervision from the force sensor and the condition flags.

Two platforms are covered: the ROBOTIS OpenManipulator-X (Dynamixel XM430-W350, 5 motors), the SO-101 (Feetech STS3215, LeRobot ecosystem, 6 joints) and the 7-DoF Franka Panda. Architecture and feature normalization are identical across platforms; only the joint count and the input feature layout differ.

• Code, configs, dataset and full documentation: github.com/Frank-ZY-Dou/Dynamics-Modeling/tree/main/NeuralActuator
• Training data: the Neural Actuation Dataset (NAD), 450 trajectories across 45 tasks on the two platforms, shipped with the code release and mirrored on the Hub.
• Project page: frank-zy-dou.github.io/projects/NeuralActuator

Checkpoints

Eleven checkpoints, one per benchmark configuration. Verify downloads against these md5 sums:

Table numbers refer to the results tables in the paper and the GitHub README, which also carries the evaluation numbers of each released checkpoint on the NAD test split. The Table 3 checkpoint ships as both the raw and the EMA weights of the same training run; the results table reports the EMA variant.

Each .pkl is a plain pickle of a dict with keys params (Flax parameter tree), ema_params (EMA copy of the same tree), and feature_mean / feature_std (normalization statistics), so evaluation needs no extra files.

How to load

import pickle
from huggingface_hub import hf_hub_download

path = hf_hub_download("frankzydou/NeuralActuator", "omx_weight_all_ema.pkl")
ckpt = pickle.load(open(path, "rb"))   # params, ema_params, feature_mean, feature_std

To run the checkpoints, use the inference and evaluation scripts from the GitHub release, which reconstruct the model around the parameter tree:

# single-trajectory rollout with predicted force
python infer_actuator.py --robot omx --checkpoint omx_weight_all_ema.pkl \
    --config configs/lift_hold.yaml \
    --csv data/weight/pick_place_object_500g/test/001.csv \
    --out outputs/pick_place_500g_pred.npz

# telemetry-only virtual force sensor (no simulator in the loop)
python infer_actuator.py --robot omx --checkpoint omx_weight_all_ema.pkl \
    --config configs/lift_hold.yaml \
    --csv data/weight/pick_place_object_500g/test/001.csv \
    --force_only --out outputs/pick_place_500g_force.npz

# full benchmark evaluation
bash scripts/eval_force_sensor.sh omx_force_sensor.pkl 0

The config only supplies the architecture and simulation settings, which all released checkpoints share, so any config for the same robot works. In the telemetry-only mode a forward pass takes about 2 ms per step on a plain CPU after JIT warmup, well inside the ~60 Hz servo telemetry rate.

License

MIT, same as the code release.

Citation

@inproceedings{dou2026neuralactuator,
  title     = {{NeuralActuator}: Neural Actuation Modeling for Robot Dynamics and External Force Perception},
  author    = {Dou, Zhiyang and Onyemelukwe, John U. and Zhang, Hangxing and Zhang, Heng and Guo, Minghao and Tian, Yunsheng and Lipiec, Michal Piotr and Jacob, Joshua and Liu, Chao and Chen, Peter Yichen and Ivanov, Yuri and Matusik, Wojciech},
  booktitle = {Proceedings of Robotics: Science and Systems (RSS)},
  year      = {2026}
}