Downloads · 30 days
13
1% of all-time downloads
distilvit
tarekziade/distilvit
distilvit is a image-to-text model from tarekziade. Use it when you need a caption or text from an image. It is set up for transformers. The card lists the license as apache-2.0.
This model is a variation of https://huggingface.co/nlpconnect/vit-gpt2-image-captioning
All-time downloads
950
Public
Parameters
182M
2.6 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.onnx1.9 GB · 72%
At a glance
- Task
- Image-to-Text
- Library
- transformers
- License
- apache-2.0
- Model type
- vision-encoder-decoder
- Access
- Public
- Created
- Mar 16, 2024
- Updated
- Mar 18, 2024
- SHA
- b5299734
Base models
Model card
From the Hugging Face model README
This model is a variation of https://huggingface.co/nlpconnect/vit-gpt2-image-captioning
- Read the blog post here https://ziade.org/2024/03/17/distilvit-image-captioning-model
- The training code is here: https://github.com/tarekziade/distilvit
Results after after 3 epochs (and ~45 hours of training)
- eval_loss: 0.19939416646957397
- eval_rouge1: 43.006
- eval_rouge2: 16.9939
- eval_rougeL: 38.8923
- eval_rougeLsum: 38.8877
- eval_gen_len: 11.327256736227712
- eval_runtime: 1816.5255
- eval_samples_per_second: 13.77
- eval_steps_per_second': 1.721
- train_runtime: 46263.3695
- train_samples_per_second: 38.373
- train_steps_per_second: 4.797
- train_loss: 0.05974134062104816
- Task
- Image-to-Text
- Library
- transformers
- Type
- vision-encoder-decoder
- License
- apache-2.0
- Created
- Mar 16, 2024
- Updated
- Mar 18, 2024