Downloads · 30 days
14
4% of all-time downloads
prosoro/prosoro-mvae
prosoro-mvae is a robotics model from prosoro. Use it for the robotics task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
Downloads · 30 days
14
4% of all-time downloads
All-time downloads
347
Public
Repo size
439 MB
Likes
0
Public
Click a slice to open those files.
.safetensors176 MB · 67%
From the Hugging Face model README
Project Page | Paper | GitHub
<div> <a href="https://hanxudong.cc">Xudong Han</a><sup>1</sup>, <a href="https://gabriel-ning.github.io">Ning Guo</a><sup>2</sup>, <a href="">Ronghan Xu</a><sup>1</sup>, <a href="https://maindl.ancorasir.com">Fang Wan</a><sup>1</sup>, <a href="https://bionicdl.ancorasir.com">Chaoyang Song</a><sup>1</sup> </br> <sup>1</sup> Southern University of Science and Technology, <sup>2</sup> Shanghai Jiao Tong University </br></br> <div align="center"> <img src="https://github.com/ancorasir/ProSoRo/blob/main/assets/img/teaser.gif?raw=true" width="80%"> </div> </div>Proprioceptive Soft Robot (ProSoRo) is a proprioceptive soft robotic system that utilizes miniature vision to track an internal marker within the robot's deformable structure. By monitoring the motion of this single point relative to a fixed boundary, we capture critical information about the robot's overall deformation state, significantly reducing sensing complexity. To harness the full potential of this anchor-based approach, we developed a multi-modal proprioception learning framework utilizing a multi-modal variational autoencoder (MVAE) to align motion, force, and shape of ProSoRos into a unified representation based on an anchored observation, involving three stages:

Within the latent code, we identify key morphing primitives that correspond to fundamental deformation modes. By systematically varying these latent components, we can generate a spectrum of deformation behaviors, offering a novel perspective on soft robotic systems' intrinsic dimensionality and controllability. This understanding enhances the interpretability of the latent code and facilitates the development of more sophisticated control strategies and advanced human-robot interfaces.

This model is intended for researchers and practitioners in the field of soft robotics who are interested in developing proprioceptive capabilities for soft robotic systems. See project page for more details.
To load the model:
# Example code to load safetensors
from transformers import AutoModel
model = AutoModel.from_pretrained("asRobotics/prosoro-mvae", prosoro_type="cylinder")
x = torch.zeros((1, 6)) # Example input: batch size of 1, 6D motion
output = model(x)
Or to load the ONNX version:
# Example code to load onnx
import onnxruntime as ort
import numpy as np
from huggingface_hub import hf_hub_download
onnx_model_path = hf_hub_download(repo_id="asRobotics/prosoro-mvae", filename="cylinder/model.onnx")
ort_session = ort.InferenceSession(onnx_model_path)
x = np.zeros((1, 6)).astype(np.float32) # Example input: batch size of 1, 6D motion
outputs = ort_session.run(None, {"motion": x})
The model was trained on the ProSoRo-100K dataset, which contains 100,000 samples of simulated data for various shapes of ProSoRos.
If you use this model in your research, please cite the following paper:
@article{han2025anchoring,
title={Anchoring Morphological Representations Unlocks Latent Proprioception in Soft Robots},
author={Han, Xudong and Guo, Ning and Xu, Ronghan and Wan, Fang and Song, Chaoyang},
journal={Advanced Intelligent Systems},
volume={0},
pages={0-0},
year={2025}
}