Downloads · 30 days
8
2% of all-time downloads
distilvit-pexels-frozen
tarekziade/distilvit-pexels-frozen
distilvit-pexels-frozen is a image-to-text model from tarekziade. Use it when you need a caption or text from an image. It is set up for transformers. The card lists the license as apache-2.0.
This model is a work in progress. Fine-tuned version of those base models:
All-time downloads
457
Public
Repo size
6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.onnx1.9 GB · 72%
At a glance
- Task
- Image-to-Text
- Library
- transformers
- License
- apache-2.0
- Model type
- vision-encoder-decoder
- Access
- Public
- Created
- Jul 30, 2024
- Updated
- Jul 31, 2024
- SHA
- 5bbea1f7
Base models
Model card
From the Hugging Face model README
distilvit
This model is a work in progress. Fine-tuned version of those base models:
- a VIT model for the image encoder: https://huggingface.co/google/vit-base-patch16-224-in21k
- a Distilled GPT-2 model for the text decoder: https://huggingface.co/distilbert/distilgpt2
This model was trained on:
- Flickr30k : https://huggingface.co/datasets/nlphuji/flickr30k
- COCO 2017: https://cocodataset.org
You can get that checkpoint using the 3083a3cef6e3c8dd90df3f088074bbe836b0f403 commit.
It was then further fine-tuned on :
For the latter, the dataset was annotated by our team to correct the alt text generated by the model, using the checkvite tool.
You can find the code used to create the model here: https://github.com/mozilla/distilvit
- Task
- Image-to-Text
- Library
- transformers
- Type
- vision-encoder-decoder
- License
- apache-2.0
- Created
- Jul 30, 2024
- Updated
- Jul 31, 2024