vit_large_patch16_384_imagenet

Model Overview

Model Summary

Vision Transformer (ViT) adapts the Transformer architecture, originally designed for natural language processing, to the domain of computer vision. It treats images as sequences of patches, similar to how Transformers treat sentences as sequences of words.. It was introduced in the paper An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.

Links:

• Vit Quickstart Notebook
• Vit API Documentation
• Vit Model Card
• KerasHub Beginner Guide
• KerasHub Model Publishing Guide

Installation

Keras and KerasHub can be installed with:

pip install -U -q keras-hub
pip install -U -q keras

Presets

Example Usage

Pretrained ViT model

image_classifier = keras_hub.models.ImageClassification.from_preset(
    "vit_large_patch16_384_imagenet"
)

input_data = np.random.uniform(0, 1, size=(2, 224, 224, 3))
image_classifier(input_data)

Load the backbone weights and fine-tune model for custom dataset.

backbone = keras_hub.models.Backbone.from_preset(
    "vit_large_patch16_384_imagenet"
)
preprocessor = keras_hub.models.ViTImageClassifierPreprocessor.from_preset(
    "vit_large_patch16_384_imagenet"
)
model = keras_hub.models.ViTImageClassifier(
    backbone=backbone,
    num_classes=len(CLASSES),
    preprocessor=preprocessor,
)

Example Usage with Hugging Face URI

Pretrained ViT model

image_classifier = keras_hub.models.ImageClassification.from_preset(
    "hf://keras/vit_large_patch16_384_imagenet"
)

input_data = np.random.uniform(0, 1, size=(2, 224, 224, 3))
image_classifier(input_data)

Load the backbone weights and fine-tune model for custom dataset.

backbone = keras_hub.models.Backbone.from_preset(
    "hf://keras/vit_large_patch16_384_imagenet"
)
preprocessor = keras_hub.models.ViTImageClassifierPreprocessor.from_preset(
    "hf://keras/vit_large_patch16_384_imagenet"
)
model = keras_hub.models.ViTImageClassifier(
    backbone=backbone,
    num_classes=len(CLASSES),
    preprocessor=preprocessor,
)