Downloads · 30 days
0
PrismCaptioner-2B
Yuxuan-Qiao/PrismCaptioner-2B
PrismCaptioner-2B is a image-text-to-text model from Yuxuan-Qiao. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for prismcaptioner. The card lists the license as cc-by-4.0.
Access
Public
Updated Jun 22, 2024
Repo size
1.1 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors1.1 GB · 100%
At a glance
- Task
- Image-Text-to-Text
- Library
- prismcaptioner
- License
- cc-by-4.0
- Access
- Public
- Created
- May 29, 2024
- Updated
- Jun 22, 2024
- SHA
- aadc2030
Try a prompt
Datasets
Model card
From the Hugging Face model README
<br>
PrismCaptioner Model Card
Model details
PrismCaptioners are open-source captioners with LLaVA architecture finetuned on GPT4V-assisted dataset ALLaVA. We have released PrismCaptioner-7B and PrismCaptioner-2B.
PrismCaptioner-2B details
- Vision Backbone: google/siglip-so400m-patch14-384
- Language Backbone: internlm/internlm2-1_8b
- Dataset: 1x ALLaVA-Caption-[LAION/VFLAN], 2x Evol-Instruct-GPT4-Turbo-143K
Paper and codebase for more information: [Paper] [Code]
Intended uses
- Perception Module: The model can be integrated into Prism as a perception module to solve vision-language task by utilizing an external LLM.
- Effective Captioner: The model can produce high-quality captions for given images.
Model usage
Clone the Prism repo and complete the preparation. You can use PrismCaptioners following usage or demo below.
# In the Prism repo folder
from decouple import supported_VLM
model = supported_VLM['prismcaptioner-2b']()
res = model.generate(['assets/case1.png', 'Given the image below, please provide a detailed description of what you see.'])
- Task
- Image-Text-to-Text
- Library
- prismcaptioner
- License
- cc-by-4.0
- Created
- May 29, 2024
- Updated
- Jun 22, 2024