Downloads · 30 days
2.6K
13% of all-time downloads
h2o-danube2-1.8b-chat-GGUF
h2oai/h2o-danube2-1.8b-chat-GGUF
h2o-danube2-1.8b-chat-GGUF is a text generation model from h2oai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
- Model creator: H2O.ai - Original model: h2o-danube2-1.8b-chat
All-time downloads
19.4K
Public
Repo size
26.3 GB
Likes
8
Public
Repo makeup
Click a slice to open those files.
.gguf26.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- May 29, 2024
- Updated
- May 29, 2024
- SHA
- 8683c1af
Try a prompt
Model card
From the Hugging Face model README
h2o-danube2-1.8b-chat-GGUF
- Model creator: H2O.ai
- Original model: h2o-danube2-1.8b-chat
Description
This repo contains GGUF format model files for h2o-danube2-1.8b-chat quantized using llama.cpp framework.
Table below summarizes different quantized versions of h2o-danube2-1.8b-chat. It shows the trade-off between size, speed and quality of the models.
| Name | Quant method | Model size | MT-Bench AVG | Perplexity | Tokens per second |
|---|---|---|---|---|---|
| h2o-danube2-1.8b-chat-F16.gguf | F16 | 3.66 GB | 5.60 | 8.02 | 797 |
| h2o-danube2-1.8b-chat-Q8_0.gguf | Q8_0 | 1.95 GB | 5.51 | 8.02 | 1156 |
| h2o-danube2-1.8b-chat-Q6_K.gguf | Q6_K | 1.50 GB | 5.51 | 8.03 | 1131 |
| h2o-danube2-1.8b-chat-Q5_K_M.gguf | Q5_K_M | 1.30 GB | 5.56 | 8.10 | 1172 |
| h2o-danube2-1.8b-chat-Q5_K_S.gguf | Q5_K_S | 1.27 GB | 5.49 | 8.12 | 1107 |
| h2o-danube2-1.8b-chat-Q4_K_M.gguf | Q4_K_M | 1.11 GB | 5.60 | 8.27 | 1162 |
| h2o-danube2-1.8b-chat-Q4_K_S.gguf | Q4_K_S | 1.06 GB | 5.59 | 8.34 | 1270 |
| h2o-danube2-1.8b-chat-Q3_K_L.gguf | Q3_K_L | 0.98 GB | 5.23 | 8.72 | 1442 |
| h2o-danube2-1.8b-chat-Q3_K_M.gguf | Q3_K_M | 0.91 GB | 4.91 | 8.81 | 1107 |
| h2o-danube2-1.8b-chat-Q3_K_S.gguf | Q3_K_S | 0.82 GB | 4.03 | 10.12 | 1103 |
| h2o-danube2-1.8b-chat-Q2_K.gguf | Q2_K | 0.71 GB | 3.03 | 12.56 | 1160 |
Columns in the table are:
- Name -- model name and link
- Quant method -- quantization method
- Model size -- size of the model in gigabytes
- MT-Bench AVG -- MT-Bench benchmark score. The score is from 1 to 10, the higher, the better
- Perplexity -- perplexity metric on WikiText-2 dataset. It's reported in a perplexity test from llama.cpp. The lower, the better
- Tokens per second -- generation speed in tokens per second, as reported in a perplexity test from llama.cpp. The higher, the better. Speed tests are done on a single H100 GPU
Prompt template
<|prompt|>Why is drinking water so healthy?</s><|answer|>
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Languages
- en
- Created
- May 29, 2024
- Updated
- May 29, 2024