Downloads · 30 days
14
2% of all-time downloads
vit-base-patch16-224-distilgpt2
tarekziade/vit-base-patch16-224-distilgpt2
vit-base-patch16-224-distilgpt2 is a image-to-text model from tarekziade. Use it when you need a caption or text from an image. It is set up for transformers. The card lists the license as apache-2.0.
This model is a work in progress. Fine-tuned version of those base models:
All-time downloads
570
Public
Parameters
182M
7.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.onnx1.9 GB · 56%
At a glance
- Task
- Image-to-Text
- Library
- transformers
- License
- apache-2.0
- Model type
- vision-encoder-decoder
- Access
- Public
- Created
- Jun 19, 2024
- Updated
- Jul 24, 2024
- SHA
- a15b0083
Base models
Model card
From the Hugging Face model README
distilvit
This model is a work in progress. Fine-tuned version of those base models:
- a VIT model for the image encoder: https://huggingface.co/google/vit-base-patch16-224-in21k
- a Distilled GPT-2 model for the text decoder: https://huggingface.co/distilbert/distilgpt2
This model was trained on:
- Flickr30k : https://huggingface.co/datasets/nlphuji/flickr30k
- COCO 2017: https://cocodataset.org
You can get that checkpoint using the 3083a3cef6e3c8dd90df3f088074bbe836b0f403 commit.
It was then further fine-tuned on :
For the latter, the dataset was annotated by our team to correct the alt text generated by the model, using the checkvite tool.
You can find the code used to create the model here: https://github.com/mozilla/distilvit
- Task
- Image-to-Text
- Library
- transformers
- Type
- vision-encoder-decoder
- License
- apache-2.0
- Created
- Jun 19, 2024
- Updated
- Jul 24, 2024