vit_large_patch16_224.augreg_in21k_ft_in1k

Vit Large Patch16 224

This repository contains LiteRT/TFLite exports of the TIMM image-classification model vit_large_patch16_224.augreg_in21k_ft_in1k.

Model Description

The model files were converted from pretrained TIMM weights published at timm/vit_large_patch16_224.augreg_in21k_ft_in1k.

Available Model Files

Quantization Schema

vit_large_patch16_224_int8_channelwise.tflite was quantized with AI Edge Quantizer's static W8A8 recipe (STATIC_WI8_AI8).

The schema is:

Calibration used real ImageNet validation images with the TIMM preprocessing flow for vit_large_patch16_224.augreg_in21k_ft_in1k. The resolved TIMM preprocessing config was {"crop_mode": "center", "crop_pct": 0.9, "input_size": [3, 224, 224], "interpolation": "bicubic", "mean": [0.5, 0.5, 0.5], "std": [0.5, 0.5, 0.5]}. When using APIs that expose raw tensor buffers, prepare the input and output using the quantization parameters stored in the model.

Runtime Compatibility

These artifacts are intended for LiteRT CPU and GPU execution.

The static INT8 channelwise artifact also AOT-compiled successfully through the LiteRT Qualcomm compiler plugin for SM8750, with the compiled model fully selected into Qualcomm NPU dispatch.

Enablement for other NPU backends is still under validation.

Intended Uses & Limitations

The model files were converted from pretrained weights from TIMM. The models may have their own licenses or terms and conditions derived from TIMM and the dataset used for training. It is your responsibility to determine whether you have permission to use the models for your use case.

Model Details

• Model Type: Image classification / feature backbone
• Model Stats:
  • Params (M): 304.3
  • GMACs: 59.7
  • Activations (M): 43.8
  • Image size: 224 x 224
• Papers:
  • How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers: https://arxiv.org/abs/2106.10270
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale: https://arxiv.org/abs/2010.11929v2
• Dataset: ImageNet-1k
• Pretrain Dataset: ImageNet-21k
• Original: https://github.com/google-research/vision_transformer

Citation

@article{steiner2021augreg,
  title={How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers},
  author={Steiner, Andreas and Kolesnikov, Alexander and and Zhai, Xiaohua and Wightman, Ross and Uszkoreit, Jakob and Beyer, Lucas},
  journal={arXiv preprint arXiv:2106.10270},
  year={2021}
}

@article{dosovitskiy2020vit,
  title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and  Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  journal={ICLR},
  year={2021}
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}