Downloads · 30 days
12
7% of all-time downloads
Llava-Phi2
Navyabhat/Llava-Phi2
Llava-Phi2 is a visual question answering model from Navyabhat. Use it for the visual question answering task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
This is a multimodal implementation of Phi2 model inspired by LlaVA-Phi.
All-time downloads
170
Public
Parameters
3.1B
12.4 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors12.4 GB · 100%
At a glance
- Task
- Visual Question Answering
- Library
- transformers
- License
- mit
- Model type
- llava_phi
- Access
- Public
- Created
- Jan 27, 2024
- Updated
- Jan 28, 2024
- SHA
- 5eba7380
Model card
From the Hugging Face model README
Model Card for Model ID
This is a multimodal implementation of Phi2 model inspired by LlaVA-Phi.
Model Details
- LLM Backbone: Phi2
- Vision Tower: clip-vit-large-patch14-336
- Pretraining Dataset: LAION-CC-SBU dataset with BLIP captions(200k samples)
- Finetuning Dataset: Instruct 150k dataset based on COCO
- Finetuned Model: Navyabhat/Llava-Phi2
Model Sources
<!-- Provide the basic links for the model. -->- Original Repository: Llava-Phi
- Paper [optional]: LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model
- Demo [optional]: Demo Link
How to Get Started with the Model
Use the code below to get started with the model.
- Clone this repository and navigate to llava-phi folder
git clone https://github.com/zhuyiche/llava-phi.git
cd llava-phi
- Install Package
conda create -n llava_phi python=3.10 -y
conda activate llava_phi
pip install --upgrade pip # enable PEP 660 support
pip install -e .
- Run the Model
python llava_phi/eval/run_llava_phi.py --model-path="RaviNaik/Llava-Phi2" \
--image-file="https://huggingface.co/Navyabhat/Llava-Phi2/resolve/main/people.jpg?download=true" \
--query="How many people are there in the image?"
Acknowledgement
This implementation is based on wonderful work done by:
LlaVA-Phi
Llava
Phi2
- Task
- Visual Question Answering
- Library
- transformers
- Type
- llava_phi
- License
- mit
- Languages
- en
- Created
- Jan 27, 2024
- Updated
- Jan 28, 2024