Downloads · 30 days
813
4% of all-time downloads
NVIDIA-Llama3-ChatQA-1.5-8B-GGUF
QuantFactory/NVIDIA-Llama3-ChatQA-1.5-8B-GGUF
NVIDIA-Llama3-ChatQA-1.5-8B-GGUF is a text generation model from QuantFactory. Use it when you need the model to write or continue text. The card lists the license as llama3.
This is Quantized version of nvidia/Llama3-ChatQA-1.5-8B created using llama.cpp
All-time downloads
18.6K
Public
Repo size
72.7 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.gguf72.7 GB · 100%
At a glance
- Task
- Text Generation
- License
- llama3
- Access
- Public
- Created
- May 5, 2024
- Updated
- May 6, 2024
- SHA
- 392deac0
Try a prompt
Base models
Model card
From the Hugging Face model README
Llama3-ChatQA-1.5-8B- GGUF
This is Quantized version of nvidia/Llama3-ChatQA-1.5-8B created using llama.cpp
Model Details
We introduce Llama3-ChatQA-1.5, which excels at conversational question answering (QA) and retrieval-augmented generation (RAG). Llama3-ChatQA-1.5 is developed using an improved training recipe from ChatQA (1.0), and it is built on top of Llama-3 base model. Specifically, we incorporate more conversational QA data to enhance its tabular and arithmetic calculation capability. Llama3-ChatQA-1.5 has two variants: Llama3-ChatQA-1.5-8B and Llama3-ChatQA-1.5-70B. Both models were originally trained using Megatron-LM, we converted the checkpoints to Hugging Face format.
Other Resources
Llama3-ChatQA-1.5-70B Evaluation Data Training Data Retriever Paper
Benchmark Results
Results in ChatRAG Bench are as follows:
| ChatQA-1.0-7B | Command-R-Plus | Llama-3-instruct-70b | GPT-4-0613 | ChatQA-1.0-70B | ChatQA-1.5-8B | ChatQA-1.5-70B | |
|---|---|---|---|---|---|---|---|
| Doc2Dial | 37.88 | 33.51 | 37.88 | 34.16 | 38.9 | 39.33 | 41.26 |
| QuAC | 29.69 | 34.16 | 36.96 | 40.29 | 41.82 | 39.73 | 38.82 |
| QReCC | 46.97 | 49.77 | 51.34 | 52.01 | 48.05 | 49.03 | 51.40 |
| CoQA | 76.61 | 69.71 | 76.98 | 77.42 | 78.57 | 76.46 | 78.44 |
| DoQA | 41.57 | 40.67 | 41.24 | 43.39 | 51.94 | 49.6 | 50.67 |
| ConvFinQA | 51.61 | 71.21 | 76.6 | 81.28 | 73.69 | 78.46 | 81.88 |
| SQA | 61.87 | 74.07 | 69.61 | 79.21 | 69.14 | 73.28 | 83.82 |
| TopioCQA | 45.45 | 53.77 | 49.72 | 45.09 | 50.98 | 49.96 | 55.63 |
| HybriDial* | 54.51 | 46.7 | 48.59 | 49.81 | 56.44 | 65.76 | 68.27 |
| INSCIT | 30.96 | 35.76 | 36.23 | 36.34 | 31.9 | 30.1 | 32.31 |
| Average (all) | 47.71 | 50.93 | 52.52 | 53.90 | 54.14 | 55.17 | 58.25 |
| Average (exclude HybriDial) | 46.96 | 51.40 | 52.95 | 54.35 | 53.89 | 53.99 | 57.14 |
Note that ChatQA-1.5 is built based on Llama-3 base model, and ChatQA-1.0 is built based on Llama-2 base model. ChatQA-1.5 used some samples from the HybriDial training dataset. To ensure fair comparison, we also compare average scores excluding HybriDial. The data and evaluation scripts for ChatRAG Bench can be found here.
Prompt Format
We highly recommend that you use the prompt format we provide, as follows:
when context is available
<pre> System: {System} {Context} User: {Question} Assistant: {Response} User: {Question} Assistant: </pre>when context is not available
<pre> System: {System} User: {Question} Assistant: {Response} User: {Question} Assistant: </pre>The content of the system's turn (i.e., {System}) for both scenarios is as follows:
<pre> This is a chat between a user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions based on the context. The assistant should also indicate when the answer cannot be found in the context. </pre>Note that our ChatQA-1.5 models are optimized for the capability with context, e.g., over documents or retrieved context.
License
The use of this model is governed by the META LLAMA 3 COMMUNITY LICENSE AGREEMENT
- Task
- Text Generation
- License
- llama3
- Languages
- en
- Created
- May 5, 2024
- Updated
- May 6, 2024