llama-3-vision-alpha

qresearch/llama-3-vision-alpha

llama-3-vision-alpha is a machine learning model from qresearch. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

projection module trained to add vision capabilties to Llama 3 using SigLIP. built by @yeswondwerr and @qtnx

Downloads · 30 days

Access

Public

Updated May 2, 2024

Repo size

43 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin43 MB · 99%

At a glance

License: apache-2.0
Access: Public
Created: Apr 25, 2024
Updated: May 2, 2024
SHA: 812f1c92

Model card

From the Hugging Face model README

FOR HF VERSION: https://huggingface.co/qresearch/llama-3-vision-alpha-hf

llama3-vision-alpha

projection module trained to add vision capabilties to Llama 3 using SigLIP. built by @yeswondwerr and @qtnx_

usage

pip install -r requirements.txt

python __main__.py -i image_path

examples

Image	Examples
<img src="assets/demo-1.jpg" width="200"/>	What is the title of this book? answer briefly<br>The title of the book is "The Little Book of Deep Learning".<br><br>Where is the person standing? answer briefly<br>The person is standing on the balcony.<br><br>Describe the image<br>The image shows a person holding a book with a cityscape visible through the window behind them. The book has a cover with a title that reads "The Little Book of Deep Learning" in bold letters.
<img src="assets/demo-2.jpg" width="200"/>	What type of food is the girl holding? answer briefly<br>A hamburger!<br><br>What color is the woman's hair? answer briefly<br>It's white!<br><br>Describe the image<br>The image is of a young girl with short, curly hair and a sweet smile, holding a giant hamburger in her hand. She's sitting at a table with a festive dinner setting, surrounded by candles and a warm glow. Her eyes are shining with excitement and contentment as she takes a big bite of the burger.

Image

Examples

What is the title of this book? answer briefly The title of the book is "The Little Book of Deep Learning". Where is the person standing? answer briefly The person is standing on the balcony. Describe the image The image shows a person holding a book with a cityscape visible through the window behind them. The book has a cover with a title that reads "The Little Book of Deep Learning" in bold letters.

What type of food is the girl holding? answer briefly A hamburger! What color is the woman's hair? answer briefly It's white! Describe the image The image is of a young girl with short, curly hair and a sweet smile, holding a giant hamburger in her hand. She's sitting at a table with a festive dinner setting, surrounded by candles and a warm glow. Her eyes are shining with excitement and contentment as she takes a big bite of the burger.

acknowledgements

Liu et al. : LLaVA
Moon et al. : AnyMAL
vikhyatk : moondream, test images

                                       .x+=:.                                                             
                                      z`    ^%                                                  .uef^"    
               .u    .                   .   <k                           .u    .             :d88E       
    .u@u     .d88B :@8c       .u       .@8Ned8"      .u          u      .d88B :@8c        .   `888E       
 .zWF8888bx ="8888f8888r   ud8888.   .@^%8888"    ud8888.     us888u.  ="8888f8888r  .udR88N   888E .z8k  
.888  9888    4888>'88"  :888'8888. x88:  `)8b. :888'8888. .@88 "8888"   4888>'88"  <888'888k  888E~?888L 
I888  9888    4888> '    d888 '88%" 8888N=*8888 d888 '88%" 9888  9888    4888> '    9888 'Y"   888E  888E 
I888  9888    4888>      8888.+"     %8"    R88 8888.+"    9888  9888    4888>      9888       888E  888E 
I888  9888   .d888L .+   8888L        @8Wou 9%  8888L      9888  9888   .d888L .+   9888       888E  888E 
`888Nx?888   ^"8888*"    '8888c. .+ .888888P`   '8888c. .+ 9888  9888   ^"8888*"    ?8888u../  888E  888E 
 "88" '888      "Y"       "88888%   `   ^"F      "88888%   "888*""888"     "Y"       "8888P'  m888N= 888> 
       88E                  "YP'                   "YP'     ^Y"   ^Y'                  "P'     `Y"   888  
       98>                                                                                          J88"  
       '8                                                                                           @%    
        `                                                                                         :"

License: apache-2.0
Languages: en
Created: Apr 25, 2024
Updated: May 2, 2024