Downloads · 30 days
679
3% of all-time downloads
flan-t5-xxl-gguf
dumb-dev/flan-t5-xxl-gguf
flan-t5-xxl-gguf is a machine learning model from dumb-dev. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as apache-2.0.
All-time downloads
23.9K
Public
Repo size
166 GB
Likes
17
Public
Repo makeup
Click a slice to open those files.
.gguf186 GB · 100%
At a glance
- Library
- llama.cpp
- License
- apache-2.0
- Access
- Public
- Created
- Oct 28, 2024
- Updated
- Oct 29, 2024
- SHA
- f795b14c
Base models
Model card
From the Hugging Face model README
flan-t5-xxl-gguf
This is a quantized version of google/flan-t5-xxl
Usage/Examples
./llama-cli -m /path/to/file.gguf --prompt "your prompt" --n-gpu-layers nn
nn --> numbers of layers to offload to gpu
Quants
| BITs | TYPE |
|---|---|
| Q2 | Q2_K |
| Q3 | Q3_K, Q3_K_L, Q3_K_M, Q3_K_S |
| Q4 | Q4_0, Q4_1, Q4_K, Q4_K_M, Q4_K_S |
| Q5 | Q5_0, Q5_1, Q5_K, Q5_K_M, Q5_K_S |
| Q6 | Q6_K |
| Q8 | Q8_0 |
Additional:
| BITs | TYPE/float |
|---|---|
| 16 | f16 |
| 32 | f32 |
Disclaimer
I don't claim any rights on this model. All rights go to google.
Acknowledgements
- Library
- llama.cpp
- License
- apache-2.0
- Languages
- de, fr, en, ro
- Created
- Oct 28, 2024
- Updated
- Oct 29, 2024