Motius-KIMODO-SMPLX-RP

<!-- This model card is synchronized from docs/model_zoo/kimodo.md by tools/sync_model_zoo_cards.py. -->

KIMODO - Kinematic Motion Diffusion

KIMODO is integrated as an motius runtime wrapper around a vendored copy of NVIDIA's official Python runtime under motius.models.motion.kimodo._vendor. Unlike the T2M-only baselines, the KIMODO pipeline is multi-task: the same bundle covers plain text generation and KIMODO's kinematic-control interfaces.

Implementation status: the artifact inference path is now ref_repo-free. KIMODOBundle imports the vendored runtime from the motius package. The SOMA-RP artifact packages the checkpoint snapshot and local text encoders; the G1 artifacts package their checkpoint snapshots and resolve the shared KIMODO text encoders from the SOMA-RP artifact on first model load.

Weights

The SOMA-RP model-zoo artifact is self-contained, including the KIMODO checkpoint snapshot and text-encoder files needed by KIMODO's local encoder. The G1 artifacts store their own KIMODO checkpoint snapshots and point text_encoders_repo at the SOMA-RP artifact to avoid duplicating the same 16GB LLM2Vec / Meta-Llama tree in every KIMODO repository.

Because the KIMODO artifacts depend on third-party text-encoder weights, their redistribution status follows the upstream KIMODO, LLM2Vec, and Meta Llama licenses. The uploaded motius repos are kept private until those redistribution terms are reviewed. The Meta-Llama text encoder is stored once in standard Transformers safetensors shards inside the SOMA-RP artifact; duplicate upstream original/*.pth exports are intentionally omitted.

Loading

Load from Hugging Face with the same from_pretrained style used by the other motius model-zoo entries:

from motius.pipelines.motion.kimodo_pipeline import KIMODOPipeline

pipe = KIMODOPipeline.from_pretrained(
    "ZeyuLing/Motius-KIMODO-SOMA-RP",
    device="cuda",
)
out = pipe.text_to_motion("a person walks forward.", num_frames=150)

Use the matching repo id for Unitree G1 or SMPL-X variants:

pipe = KIMODOPipeline.from_pretrained(
    "ZeyuLing/Motius-KIMODO-G1-RP",  # or motius-kimodo-g1-seed
    device="cuda",
)
out = pipe.text_to_motion("a humanoid robot walks forward.", num_frames=150)

pipe = KIMODOPipeline.from_pretrained(
    "ZeyuLing/Motius-KIMODO-SMPLX-RP",
    device="cuda",
)
out = pipe.text_to_motion("a person walks forward.", num_frames=150)

For G1 and SMPLX artifacts, KIMODOBundle.from_pretrained resolves the shared text_encoders/ tree from ZeyuLing/Motius-KIMODO-SOMA-RP when the model is loaded. Since these repos are private during license review, run with an authenticated Hugging Face token that can read the KIMODO motius repos.

The lower-level bundle is still available when you need direct access to KIMODO runtime paths:

from motius.models.motion.kimodo import KIMODOBundle
from motius.pipelines.motion.kimodo_pipeline import KIMODOPipeline

bundle = KIMODOBundle.from_pretrained(
    "ZeyuLing/Motius-KIMODO-SOMA-RP",
    device="cuda",
)
pipe = KIMODOPipeline(bundle)

For environments that need an explicit local snapshot:

from huggingface_hub import snapshot_download
from motius.models.motion.kimodo import KIMODOBundle

path = snapshot_download("ZeyuLing/Motius-KIMODO-SOMA-RP")
bundle = KIMODOBundle.from_pretrained(path, device="cuda")

Local construction from official NVIDIA checkpoint folders is still available:

from motius.models.motion.kimodo import KIMODOBundle

bundle = KIMODOBundle.from_pretrained(
    "Kimodo-SOMA-RP-v1",
    device="cuda",
    text_encoder_mode="local",
    checkpoint_dir="checkpoints/kimodo/local_models",
    text_encoders_dir="checkpoints/kimodo/text_encoders",
)

Supported Tasks

The wrapper exposes the task surface KIMODO itself supports:

Examples:

# Multi-prompt stitching.
out = pipe.multi_prompt(
    ["a person walks forward.", "the person turns left."],
    [90, 90],
)

# Constraint JSON produced by the official KIMODO demos.
constraints = pipe.constraints_from_json("constraints.json", device="cuda")
out = pipe.constrained_motion(
    "a person follows the specified path.",
    num_frames=180,
    constraints=constraints,
)

# Programmatic 2D root waypoints.
root2d = pipe.root2d_constraint(
    frame_indices=[0, 60, 120],
    smooth_root_2d=[[0.0, 0.0], [0.8, 0.1], [1.5, 0.6]],
    device="cuda",
)
out = pipe.constrained_motion(
    "a person walks along a curved path.",
    num_frames=150,
    constraints=[root2d],
)

Retargeting To SMPL

For mesh inspection and HumanML3D-style evaluation, retarget KIMODO SOMA output through the rotation-aware SOMA-to-SMPL operator:

from motius.motion.retarget import KIMODOSOMAToSMPLRetargeter

retargeter = KIMODOSOMAToSMPLRetargeter(device="cuda")
smpl = retargeter.retarget_file("kimodo_debug_npz/000000.npz")
motion_135 = smpl["motion_135"]

The debug NPZ must contain KIMODO global_rot_mats. Position-only IK is a fallback for diagnostics and should not be used as the mesh-quality path.

Artifact Format

KIMODOBundle.save_pretrained() stores a complete runtime artifact:

kimodo_config.json
model_index.json
README.md
kimodo_checkpoint/Kimodo-SOMA-RP-v1/
text_encoders/

G1 and SMPLX artifacts use the same format but omit text_encoders/ locally:

kimodo_config.json
model_index.json
README.md
kimodo_checkpoint/Kimodo-G1-RP-v1/

KIMODOBundle.from_pretrained(path_or_repo) resolves the artifact-local kimodo_checkpoint/ and text_encoders/ directories into KIMODO's CHECKPOINT_DIR and TEXT_ENCODERS_DIR environment hooks before calling the official loader.

Package the default SOMA-RP model:

python3 scripts/eval/convert_kimodo_checkpoint.py \
  --model_name Kimodo-SOMA-RP-v1 \
  --checkpoint_dir checkpoints/kimodo/local_models \
  --text_encoders_dir checkpoints/kimodo/text_encoders \
  --out_dir checkpoints/kimodo/motius_soma_rp \
  --verify

Package a G1 checkpoint from a local Hugging Face snapshot:

python3 scripts/eval/convert_kimodo_checkpoint.py \
  --model_name Kimodo-G1-RP-v1 \
  --checkpoint_source checkpoints/kimodo/hub/models--nvidia--Kimodo-G1-RP-v1/snapshots/<sha> \
  --out_dir checkpoints/kimodo/motius_g1_rp \
  --no_text_encoder \
  --text_encoders_repo ZeyuLing/Motius-KIMODO-SOMA-RP \
  --copy_mode hardlink \
  --verify

Evaluation Status

Current status is intentionally split by task. The trusted HumanML3D T2M numbers below are from the SMPL-X-RP motius artifact, exported through the SMPL-X/SOMA-to-SMPL bridge and then scored with the persisted motius evaluators.

HumanML3D T2M Metrics (SMPL-X-RP, 2026-06-16)

Prediction/evaluation root: outputs/evaluation/kimodo_smplx_hml3d_smpl_ms272_v100x1_20260616.

The MotionStreamer-272 score uses the explicit KIMODO SMPL-X output -> SMPL motion_135 -> MotionStreamer-272 conversion chain. The GT(real) sanity row in the same run matches the persisted MotionStreamer evaluator contract (R@1=0.706, Diversity≈27.36, MM-Dist≈15.01), so these numbers are suitable for the current model card.

Reproduce the run with the Taiji submit helper:

python3 scripts/submit/submit_kimodo_hml3d_smpl_ms272_taiji.py \
  --out-root outputs/evaluation/kimodo_smplx_hml3d_smpl_ms272_v100x1_20260616 \
  --feature-namespace kimodo_smplx_t2m_hml3d_smpl_ms272_20260616 \
  --num-jobs 24 \
  --gpus-per-job 1 \
  --no-submit-cache

Visualization Protocol

KIMODO visualization is defined by reusable motius motion protocols, not by a one-off web page. The task/panel/frame contracts live in motius.motion.visualization.kimodo and motius.motion.visualization.protocol; the detailed condition families are documented in docs/motion/kimodo_visualization_protocols.md.

KIMODO is not a single T2M viewer case. Each supported task must expose the generated motion plus the task condition that the model was asked to satisfy. The current manifest layout is:

outputs/evaluation/<kimodo-viewer-root>/
  _manifest.json              # protocol rows from motius.motion.visualization
  _captions.json
  gt/<case>.npz               # same-timeline reference / condition-source SMPL motion
  condition_smpl/<case>.npz   # generated-timeline condition source, visible only on constrained frames
  kimodo_smpl/<case>.npz      # generated KIMODO retargeted to SMPL
  kimodo_soma/<case>.npz      # generated native SOMA mesh

_manifest.json includes frame_semantics, condition_overlays, panel_visible_ranges, missing_panels, and diagnostics. Missing panels are source/export limitations, not UI bugs. A gt panel is only valid when the reference motion is finite, nonzero, and on the same frame timeline as the generated motion. For transition/prepend tasks, use condition_smpl plus panel_visible_ranges.condition_smpl instead of fabricating full-timeline GT.

Run the viewer:

PYTHONPATH=$PWD HFTRAINER_SKIP_AUTOREGISTER=1 \
python3 motion_annot_web/m2m_eval_viewer/retarget_smpl_app.py \
  --host 0.0.0.0 --port 8216 \
  --root outputs/evaluation/kimodo_all_tasks_mesh_viewer_20260615_refactor \
  --model gt=gt=GT=#2ca02c \
  --model condition=condition_smpl=Condition-SMPL=#f59e0b \
  --model soma=kimodo_soma=KIMODO-SOMA=#f97316 \
  --model smpl=kimodo_smpl=KIMODO-SMPL=#9467bd \
  --case-mode union --color-mode condition --list-captions

Current live debug URL on the workstation uses port 80:

http://21.6.58.73/

Task protocols:

Condition frames are markers or overlays on the generated sequence, not a separate stream of motion. If the mesh visibly jumps around a keyframe marker, that is not an expected viewer-side transition. Check diagnostics.continuity in the manifest and then debug the KIMODO constraint / retarget export for that sample.

The viewer consumes _manifest.json instead of guessing from directory names. Each panel has an explicit role (reference, generated, generated_native), and each case has frame_semantics so discrete keyframes/endpoints and continuous-control tasks are displayed differently.

Relevant scripts:

• scripts/submit/submit_kimodo_hml3d_smpl_ms272_taiji.py submits the current KIMODO T2M -> SOMA -> SMPL -> HumanML3D/MS272 path.
• scripts/analysis/build_kimodo_task_mesh_viewer.py assembles a compact task-protocol-aware GT / KIMODO-SMPL / optional KIMODO-SOMA visual sanity fixture for the supported KIMODO task surface.
• scripts/submit/submit_kimodo_t2m_eval.py keeps the older T2M submission entry point.
• scripts/kimodo/run_kimodo_all_tasks.py and tools/run_kimodo_all_tasks.py cover the broader KIMODO task family.
• scripts/eval/run_kimodo_tp2m_table2.sh covers prefix-pose + text generation.
• scripts/eval/run_e10_kimodo_batch.sh and scripts/eval/run_e10_kimodo_h3d500_metrics.sh cover the E10 bridge path.

Direct Loading

from motius import Pipeline

pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-KIMODO-SMPLX-RP")