figure-skating-gcn

Figure Skating GCN Action Classifier

A spatial Graph Convolutional Network stem + Transformer-encoder model for classifying figure-skating actions (jumps, spins, sequences) from pose-skeleton time series. Treats the 17 COCO joints as a graph (edges = anatomical bones), applies Kipf-Welling style graph convolution per frame, then a Transformer encoder does the temporal mixing across frames.

This repo holds two checkpoints trained on the same architecture and data split, differing only in label granularity:

Not transformers-compatible in the HF sense — this is a plain PyTorch nn.Module + state_dict checkpoint (custom architecture, not a Hub AutoModel). Model source code (SkatingGCNClassifier in the project's model_gcn.py, which depends on model.py, model_transformers.py, and preprocessing.py) is not included in this repo — you'll need the project source to reconstruct the class before loading the state dict.

How to load

from huggingface_hub import hf_hub_download
import torch

repo_id = "Mercity/figure-skating-gcn"
ckpt_path = hf_hub_download(repo_id, "model_gcn_fine.pt")
ckpt = torch.load(ckpt_path, map_location="cpu", weights_only=False)

# reconstruct with the project's SkatingGCNClassifier (model_gcn.py):
# model = SkatingGCNClassifier(
#     ckpt["num_classes"], node_features=ckpt["node_features"], gcn_layers=ckpt["gcn_layers"],
# )
# model.load_state_dict(ckpt["state_dict"])
# model.eval()

# ckpt also carries the standardization stats used at train time and the taxonomy:
mu, sd = ckpt["feature_mean"], ckpt["feature_std"]     # normalize inputs with (x - mu) / sd
taxonomy = ckpt["taxonomy"]                            # {0: "3Toeloop", 1: "3Loop", ...}

Swap model_gcn_fine.pt for model_gcn_coarse.pt to load the 11-class variant.

Config

Data

Trained on skeleton-extracted figure-skating clips (YOLO11n-pose, COCO-17 joints, imgsz=384) from an internal Mercity dataset. Stratified train/val/test split, seed 42. Not included in this repo (private internal dataset).

Results (test split)

Fine-grained (28-class)

Early-stopped epoch 64 (best val macro-F1 0.517)

Overall: accuracy 0.8278 · precision (macro/weighted) 0.4863 / 0.8465 · recall 0.4581 / 0.8278 · F1 0.4473 / 0.8266

Coarse (11-class)

Early-stopped epoch 94 (best val macro-F1 0.845)

Overall: accuracy 0.9039 · precision (macro/weighted) 0.8403 / 0.9068 · recall 0.8456 / 0.9039 · F1 0.8393 / 0.9036

Notable failure modes (from confusion matrix analysis, coarse label space)

• Second-best coarse macro-F1 of all architectures tested (0.839, behind only CNN-BiLSTM's 0.851) — the spatial graph structure pays off despite double the parameter count of the simpler CNN-BiLSTM baseline.
• Weakest fine-grained result among the "full-feature" models (0.447 macro-F1) — the per-frame spatial graph convolution doesn't help much once the task is telling 3Flip from 3Lutz apart, which is primarily a fine temporal-motion distinction rather than a spatial one.
• Flip ↔ Lutz confusion present, consistent with every other architecture tested.
• Fine-grained macro-F1 is dragged down almost entirely by 1-2-sample tail classes (4Flip, 4Lutz, 4Loop, 1Salchow, 1Toeloop, 1Flip, 1Lutz) — a data scarcity issue, not an architecture weakness.

License

Internal/proprietary — not licensed for external use. Contact the Mercity team for access terms.