Downloads · 30 days
48
2% of all-time downloads
Yi-VL-6B-GGUF
cmp-nct/Yi-VL-6B-GGUF
Yi-VL-6B-GGUF is a machine learning model from cmp-nct. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a high quality quantization of Yi-VL-6B and of the visual transformer. Q5K is almost equal to fp16 in inference and Q6K is basically the same (not as reliably tested for visual transformers but I assume it's e…
All-time downloads
3K
Public
Repo size
4.8 GB
Likes
7
Public
Repo makeup
Click a slice to open those files.
.gguf4.8 GB · 100%
At a glance
- Access
- Public
- Created
- Jan 24, 2024
- Updated
- Jan 24, 2024
- SHA
- 72140010
Model card
From the Hugging Face model README
This is a high quality quantization of Yi-VL-6B and of the visual transformer. Q5_K is almost equal to fp16 in inference and Q6_K is basically the same (not as reliably tested for visual transformers but I assume it's equal to language models) You currently need to apply this PR to make it work: https://github.com/ggerganov/llama.cpp/pull/5093 - this adds the additional normalization steps into the projection
I do not like this model, it's hallucinating more than anything else based on llava.
- Created
- Jan 24, 2024
- Updated
- Jan 24, 2024