Motius-MoGenTS-HumanML3D

<!-- This model card is synchronized from docs/model_zoo/mogents.md by tools/sync_model_zoo_cards.py. -->

MoGenTS - Motion Generation Based on Spatial-Temporal Joint Modeling

Text-to-motion baseline integrated into the motius Model Zoo. The runtime is self-contained under motius.models.motion.mogents.network and does not import the original repository at inference time.

---

Weights

Self-contained motius artifact:

Convert the official checkpoints into a self-contained motius artifact:

python3 scripts/eval/convert_mogents_checkpoint.py \
    --weights_root logs \
    --length_root checkpoints \
    --out_dir checkpoints/mogents/humanml3d \
    --verify

Expected artifact layout:

checkpoints/mogents/humanml3d/
  mogents_config.json
  model_index.json
  vq.safetensors
  mask_aux.safetensors
  mask_ts.safetensors
  res_aux.safetensors
  res_ts.safetensors
  length_est.safetensors
  clip.safetensors
  Mean.npy
  Std.npy

Use

from motius.pipelines.mogents import MoGenTSPipeline

pipe = MoGenTSPipeline.from_pretrained(
    "ZeyuLing/Motius-MoGenTS-HumanML3D",
    device="cuda",
)
motions = pipe.infer_t2m(
    ["a person walks forward then turns around"],
    [120],
)  # list of (T, 263)

For a local mirror:

pipe = MoGenTSPipeline.from_pretrained("checkpoints/mogents/humanml3d", device="cuda")

Motion Representation

MoGenTS natively generates HumanML3D-263 at 20 fps. For cross-model comparison with SMPL or MotionStreamer-272 methods, first generate the native 263-dim outputs and then use the validated bridge:

HumanML3D-263 -> SMPL motion_135 via IK refine-80 -> MotionStreamer-272

The bridge is a representation-conversion diagnostic. It should not be treated as the native MoGenTS paper metric space.

---

Evaluation

Generate under the official HumanML3D test protocol and score with the HumanML3D-263 evaluator:

python3 scripts/eval/mogents_t2m_h3d263.py \
    --model_path checkpoints/mogents/humanml3d \
    --out_dir outputs/evaluation/t2m/humanml3d_official_test/hml263/mogents_ts10_cfg4_rescfg5_seed0

python3 scripts/eval/verify_evaluators.py --which hml263 \
    --hml263-pred outputs/evaluation/t2m/humanml3d_official_test/hml263/mogents_ts10_cfg4_rescfg5_seed0 \
    --out-dir outputs/evaluation/t2m/humanml3d_official_test/hml263/mogents_ts10_cfg4_rescfg5_seed0/metrics

HumanML3D-263 evaluator (native space, n=3970)

Metric JSON: outputs/evaluation/t2m/humanml3d_official_test/hml263/mogents_ts10_cfg4_rescfg5_seed0/metrics/verify_hml263.json.

SMPL motion_135 + MotionStreamer-272 evaluator

Convert the same HumanML3D test predictions to SMPL motion_135 and then to MotionStreamer-272:

NUM_GPUS=8 NUM_SHARDS=8 N_REPEATS=20 \
    bash scripts/eval/run_mogents_hml263_to_ms272_chain.sh

The restartable script runs the following stages with --skip-existing:

python3 scripts/eval/hml263_to_smpl_ik.py \
    --in-dir outputs/evaluation/t2m/humanml3d_official_test/hml263/mogents_ts10_cfg4_rescfg5_seed0 \
    --out-dir outputs/evaluation/t2m/humanml3d_official_test/motion135/mogents_ts10_cfg4_rescfg5_seed0_ik80 \
    --model-dir ref_repo/MDM/body_models \
    --source-fps 20 --target-fps 30 \
    --floor-align --refine-iters 80 --refine-lr 0.02 \
    --device cuda --skip-existing

python3 scripts/data/convert_motion135_to_h3d272.py \
    --in-dir outputs/evaluation/t2m/humanml3d_official_test/motion135/mogents_ts10_cfg4_rescfg5_seed0_ik80 \
    --out-dir outputs/evaluation/t2m/humanml3d_official_test/ms272/mogents_ts10_cfg4_rescfg5_seed0_ik80 \
    --workers 8 --skip-existing

python3 scripts/eval/verify_evaluators.py --which ms272 \
    --ms272-pred outputs/evaluation/t2m/humanml3d_official_test/ms272/mogents_ts10_cfg4_rescfg5_seed0_ik80 \
    --n-repeats 20 \
    --out-dir outputs/evaluation/t2m/humanml3d_official_test/ms272/mogents_ts10_cfg4_rescfg5_seed0_ik80/metrics

Metric JSON: outputs/evaluation/t2m/humanml3d_official_test/ms272/mogents_ts10_cfg4_rescfg5_seed0_ik80/metrics/verify_ms272.json.

Run summary: outputs/evaluation/t2m/humanml3d_official_test/ms272/mogents_ts10_cfg4_rescfg5_seed0_ik80/metrics/run_summary.json.

Bridge outputs contain 4012 HML263 predictions, 4012 SMPL motion_135 files, and 4012 MotionStreamer-272 files. The SMPL IK shard summaries report zero conversion failures and mean joint-fit MPJPE around 15.2-15.6 mm.

Implementation Notes

• Architecture: MoGenTS generates a 1D auxiliary token stream and a 2D spatial-temporal token grid, then decodes both streams together with the dual RVQ-VAE.
• Runtime package: motius/models/motion/mogents/network/ contains only the inference-time model components from the MIT-licensed upstream code.
• Artifact loading: MoGenTSBundle.from_pretrained consumes local/HF-style artifacts and stores CLIP once as clip.safetensors; raw upstream .tar checkpoints are supported only through explicit converter/debug paths.
• Native representation: generated outputs are HumanML3D-263. Any MotionStreamer-272 or SMPL motion_135 comparison should be produced by the existing representation-conversion pipeline after generation.

Direct Loading

from motius import Pipeline

pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-MoGenTS-HumanML3D")