video-swin-base-clothes-fold-5class

<!-- This is a Hugging Face model card template. Replace the TODO markers with your final values before publishing. All figures below are pulled from the repository's evaluation reports and dataset statistics. -->

Video Swin-Base for Clothes-Fold 5-Class Action Recognition

A Video Swin Transformer Base backbone with an I3D head trained to recognize 5 sequential actions in a clothes-folding task from top-down (cam_high) video. This is the best-performing checkpoint from the project's experiment series.

• Model family: Video Swin Transformer (Base) + I3D Head
• Task: 5-class video action recognition
• Framework: mmaction2 (v0.15.0 fork)
• Best checkpoint: epoch_50.pth → Test Top-1 0.9471

Model Overview

The model classifies a short video clip into one of 5 clothes-folding actions. It follows the standard two-stage video architecture: a 3D-tokenized Swin Transformer backbone produces spatio-temporal features, and an I3D-style head pools them into a per-class score.

Model Details

Class Labels

* frist_fold is kept as-is to match the original dataset annotation spelling.

Intended Use & Limitations

Intended use: recognizing the 5 sequential stages of a clothes-folding action from top-down monocular video (e.g. a robot or human folding a garment in front of a high-mounted camera).

Limitations:

• Trained only on cam_high (top-down) viewpoints of a single cloth-folding setup.
• Not tested on other camera angles, backgrounds, garment types, or unseen folding styles.
• Data is class-balanced per split but comes from a single recording environment.

Training Data

Dataset: clothes_fold_5 — 742 episodes / 3,707 clips, split train:val:test = 12:1:1.

Labels are nearly class-balanced (≈636 train / 53 val / 53 test per class). Source data: top-down (cam_high) RGB video segmented per action.

Training Procedure

Configuration: 005_clip48_lr8e4_wd005_drop03.py

Implementation note: the mmaction2 DistOptimizerHook (gradient accumulation) is used with native fp32; mmdet fp16 is intentionally disabled for this Swin config.

Evaluation

Test protocol: num_clips=4 + ThreeCrop, average_clips=prob, max_testing_views=2.

Best Checkpoint (epoch_50.pth)

Comparison with Baselines (test Top-1)

Key findings from training:

• Longer clips (clip_len 48, frame_interval 1) + lower LR (8e-4) + weight_decay 0.05 + drop_path 0.3 give ~+2.4% over the baselines.
• Test accuracy exceeds val for all runs, so val should not be the sole model-selection signal.

How to Get Started

Below is an example using the mmaction2 test script. Place the downloaded checkpoint at the path given in --checkpoint (e.g. work_dirs/005_clip48_lr8e4_wd005_drop03/epoch_50.pth).

# Single-GPU inference / evaluation with a config + checkpoint
python tools/test.py \
  configs/recognition/swin/005_clip48_lr8e4_wd005_drop03.py \
  work_dirs/005_clip48_lr8e4_wd005_drop03/epoch_50.pth \
  --out result/epoch_50.pkl \
  --eval top_k_accuracy mean_class_accuracy

Data preprocessing / reproduction is handled by tools/prepare_clothes_fold_dataset.py; the label map lives in demo/label_map_clothes_fold.txt.

Files in This Repository

Checkpoints are full training states (~1.1 GB each) and are uploaded alongside this card.

Citation / Acknowledgements

If you use this model in research, please cite:

@misc{clothesfold-videoswin,
  title  = {Video Swin-Base for 5-Class Clothes-Fold Action Recognition},
  author = {CaoJiahao2},
  year   = {2026},
  note   = {mmaction2 (v0.15.0 fork) fine-tune of ImageNet-22K Swin-Base},
  howpublished = {\url{<this HF repo URL>}}  % TODO
}

Backbone pretraining: Liu et al., "Video Swin Transformer" / Swin Transformer (ImageNet-22K). Frameworks: mmaction2, Swin Transformer.