hftrainer-motiongpt-humanml3d

<!-- This model card is synchronized from docs/model_zoo/motiongpt.md by tools/sync_model_zoo_cards.py. -->

MotionGPT - Human Motion as a Foreign Language

Text-to-motion baseline integrated into the hftrainer Model Zoo. The runtime is self-contained under hftrainer.models.motion.motiongpt.network and does not import the original repository at inference time.

---

Weights

Self-contained hftrainer artifact:

Use directly from the Hub:

from hftrainer.pipelines.motiongpt import MotionGPTPipeline

pipe = MotionGPTPipeline.from_pretrained(
    "ZeyuLing/hftrainer-motiongpt-humanml3d",
    bundle_kwargs={"local_files_only": False},
    device="cuda",
)
motions = pipe.infer_t2m(
    ["a person walks forward then sits down"],
    [120],
)  # list of (T, 263)

For a local mirror:

pipe = MotionGPTPipeline.from_pretrained(
    "checkpoints/baselines/motiongpt",
    bundle_kwargs={"local_files_only": True},
    device="cuda",
)

Motion Representation

MotionGPT natively generates HumanML3D-263 at 20 fps. For shared SMPL and MotionStreamer-272 evaluation, use the validated bridge:

HumanML3D-263 -> SMPL motion_135 via IK refine-80 -> MotionStreamer-272

The artifact packages the released MotionGPT checkpoint, HumanML3D statistics, and the local FLAN-T5-base tokenizer/config files required to instantiate the language model without a separate upstream checkout.

HumanML3D Leaderboard Metrics

The row below uses the shared HumanML3D official-test caption protocol and the HML263 round-trip GT reference for SMPL-based evaluators. MotionCLIP metrics use raw projection embeddings without L2 normalization.

Physical metrics:

Implementation Notes

• Artifact inference imports only hftrainer.models.motion.motiongpt.network.
• The released checkpoint has FLAN-T5-base / T5-v1.1 FFN shapes rather than ordinary t5-base FFN shapes.
• The checkpoint stores a distinct LM head while sharing the encoder and decoder input embeddings; the bundle keeps shared_encoder_decoder_untied_lm_head.
• The validated HumanML3D setting uses the official no-length prompt mode (official_nolen) and the selected-caption official-test protocol.