figure-skating-ctm

Figure Skating CTM (Continuous Thought Machine) Action Classifiers

Continuous Thought Machine architecture for classifying figure-skating actions (jumps, spins, sequences) from pose-skeleton time series: a Conv1D backbone feeding a CTM "thinking loop" (recurrent neuron model + synchronization-based attention over the backbone features), run for a fixed or adaptively-halted number of steps before classification.

This repo holds four checkpoints — two thinking-step configurations, each trained on both label granularities:

Not transformers-compatible — plain PyTorch nn.Module + state_dict checkpoints. Model source code (SkatingCTM in the project's model_ctm.py, which depends on ctm_components.py, model.py, and model_gcn.py) is not included in this repo — you'll need the project source to reconstruct the class before loading the state dict.

How to load

from huggingface_hub import hf_hub_download
import torch

repo_id = "Mercity/figure-skating-ctm"
ckpt_path = hf_hub_download(repo_id, "model_ctm-10_fine.pt")
ckpt = torch.load(ckpt_path, map_location="cpu", weights_only=False)

# reconstruct with the project's SkatingCTM (model_ctm.py):
# model = SkatingCTM(
#     ckpt["in_features"], ckpt["num_classes"],
#     max_iterations=ckpt["max_iterations"], memory_length=ckpt["memory_length"],
#     use_act=ckpt["use_act"], backbone=ckpt["backbone"],
#     node_features=ckpt["node_features"], gcn_layers=ckpt["gcn_layers"],
# )
# model.load_state_dict(ckpt["state_dict"])
# model.eval()

# ckpt also carries the standardization stats used at train time and the taxonomy:
mu, sd = ckpt["feature_mean"], ckpt["feature_std"]     # normalize inputs with (x - mu) / sd
taxonomy = ckpt["taxonomy"]                            # {0: "3Toeloop", 1: "3Loop", ...}

Swap the filename to load any of the other three variants.

Config

Shared across all four checkpoints:

Where CTM-10 vs CTM-10+ACT differ:

⚠️ Note: in both fine and coarse CTM-10+ACT runs, test-time mean thinking steps measured 10.0 — every sample used the full step budget; no sample ever halted early. With a max of 10 iterations and a halt bias initialized to favor ~20 steps, ACT never got the chance to demonstrate adaptive depth in these checkpoints — treat CTM-10+ACT here as "CTM-10 + a ponder-cost regularizer on a fixed-depth computation," not genuinely adaptive-depth inference.

Data

Trained on skeleton-extracted figure-skating clips (YOLO11n-pose, COCO-17 joints, imgsz=384) from an internal Mercity dataset. Stratified train/val/test split, seed 42. Not included in this repo (private internal dataset).

Results (test split)

CTM-10 — Fine-grained (28-class)

Early-stopped epoch 99 (best val macro-F1 0.638) · test mean thinking steps: 10.0

Overall: accuracy 0.8799 · precision (macro/weighted) 0.5055 / 0.8756 · recall 0.4962 / 0.8799 · F1 0.4948 / 0.8749

CTM-10 — Coarse (11-class)

Early-stopped epoch 96 (best val macro-F1 0.849) · test mean thinking steps: 10.0

Overall: accuracy 0.8929 · precision (macro/weighted) 0.8188 / 0.8977 · recall 0.8276 / 0.8929 · F1 0.8161 / 0.8925

CTM-10+ACT — Fine-grained (28-class)

Early-stopped epoch 66 (best val macro-F1 0.550) · test mean thinking steps: 10.0

Overall: accuracy 0.8559 · precision (macro/weighted) 0.4840 / 0.8645 · recall 0.5175 / 0.8559 · F1 0.4775 / 0.8573

CTM-10+ACT — Coarse (11-class)

Early-stopped epoch 57 (best val macro-F1 0.823) · test mean thinking steps: 10.0

Overall: accuracy 0.8979 · precision (macro/weighted) 0.8201 / 0.9012 · recall 0.8362 / 0.8979 · F1 0.8262 / 0.8986

Notable failure modes (from confusion matrix analysis, coarse label space)

• Flip ↔ Lutz is the dominant confusion in both variants — edge jumps, genuinely hard to distinguish from skeleton motion alone (mirrors real judging difficulty).
• CTM-10 (fixed) shows the field's worst Salchow recall (56.7%) and Lutz recall (58.0%), plus unusually strong SitSpin→NoBasic leakage (9/132) not seen in other architectures.
• CTM-10+ACT improves slightly over CTM-10 on most coarse classes but never surpasses CNN-BiLSTM or GCN on macro-F1.
• Fine-grained macro-F1 in both variants is dragged down almost entirely by 1-2-sample tail classes (4Flip, 4Lutz, 4Loop, 1Salchow, 1Toeloop, 1Flip, 1Lutz) — a data scarcity issue, not an architecture weakness.

License

Internal/proprietary — not licensed for external use. Contact the Mercity team for access terms.