Downloads · 30 days
23
5% of all-time downloads
video-swin-transformer
Tonic/video-swin-transformer
video-swin-transformer is a machine learning model from Tonic. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for tf-keras. The card lists the license as other.
Collection of Video Swin Transformers feature extractor models.
All-time downloads
503
Public
Repo size
906 MB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.data-00000-of-00001473 MB · 52%
At a glance
- Library
- tf-keras
- License
- other
- Access
- Public
- Created
- Apr 23, 2024
- Updated
- Jul 8, 2024
- SHA
- cb47b6c0
Model card
From the Hugging Face model README
Collection shoaib6174/video_swin_transformer/1
Collection of Video Swin Transformers feature extractor models.
<!-- task: video-feature-extraction -->Overview
This collection contains different Video Swin Transformer [1] models. The original model weights are provided from [2]. There were ported to Keras models
(tf.keras.Model) and then serialized as TensorFlow SavedModels. The porting steps are available in [3].
About the models
These models can be directly used to extract features from videos. These models are accompanied by Colab Notebooks with fine-tuning steps for action-recognition task and video-classification.
The table below provides a performance summary:
| model_name | pre-train dataset | fine-tune dataset | acc@1(%) | acc@5(%) |
|---|---|---|---|---|
| swin_tiny_patch244_window877_kinetics400_1k | ImageNet-1K | Kinetics 400(1k | 78.8 | 93.6 |
| swin_small_patch244_window877_kinetics400_1k | ImageNet-1K | Kinetics 400(1k) | 80.6 | 94.5 |
| swin_base_patch244_window877_kinetics400_1k | ImageNet-1K | Kinetics 400(1k) | 80.6 | 96.6 |
| swin_base_patch244_window877_kinetics400_22k | ImageNet-12K | Kinetics 400(1k) | 82.7 | 95.5 |
| swin_base_patch244_window877_kinetics600_22k | ImageNet-1K | Kinetics 600(1k) | 84.0 | 96.5 |
| swin_base_patch244_window1677_sthv2 | Kinetics 400 | Something-Something V2 | 69.6 | 92.7 |
These scores for all the models are taken from [2].
Video Swin Transformer Feature extractors Models
- swin_tiny_patch244_window877_kinetics400_1k
- swin_small_patch244_window877_kinetics400_1k
- swin_base_patch244_window877_kinetics400_1k
- swin_base_patch244_window877_kinetics400_22k
- swin_base_patch244_window877_kinetics600_22k
- swin_base_patch244_window1677_sthv2
Notes
The input shape for these models are [None, 3, 32, 224, 224] representing [batch_size, channels, frames, height, width]. To create models with different input shape use this notebook.
References
[1] Video Swin Transformer Ze et al. [2] Video Swin Transformers GitHub [3] GSOC-22-Video-Swin-Transformers GitHub
Acknowledgements
- Library
- tf-keras
- License
- other
- Created
- Apr 23, 2024
- Updated
- Jul 8, 2024