dragon-llama-3.1-gguf

DRAGON-LLAMA-3.1-GGUF

<!-- Provide a quick summary of what the model is/does. -->

dragon-llama-3.1-gguf is RAG-instruct trained on top of a Llama-3.1 base model.

Benchmark Tests

Evaluated against the benchmark test: RAG-Instruct-Benchmark-Tester
1 Test Run (temperature=0.0, sample=False) with 1 point for correct answer, 0.5 point for partial correct or blank / NF, 0.0 points for incorrect, and -1 points for hallucinations.

--Accuracy Score: 94.0 correct out of 100
--Not Found Classification: 70.0%
--Boolean: 90.0%
--Math/Logic: 72.5%
--Complex Questions (1-5): 4 (Above Average - table-reading, causal)
--Summarization Quality (1-5): 4 (Above Average)
--Hallucinations: No hallucinations but a few instances of drawing on 'background' knowledge.

For test run results (and good indicator of target use cases), please see the files ("core_rag_test" and "answer_sheet" in this repo).

The inference accuracy tests were performed on this model (GGUF 4_K_M) not the original Pytorch, and it is possible that the original Pytorch may score higher, but we have chosen to use the quantized version as it is most representative of the likely use of the model for inference.

Please compare with dragon-llama2-7b or the most recent dragon-mistral-0.3.

Model Description

<!-- Provide a longer summary of what this model is. -->

• Developed by: llmware
• Model type: Llama-8b-3.1-Base
• Language(s) (NLP): English
• License: Llama-3.1 Community License
• Finetuned from model: Llama-3.1-Base

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

Any model can provide inaccurate or incomplete information, and should be used in conjunction with appropriate safeguards and fact-checking mechanisms.

How to Get Started with the Model

To pull the model via API:

from huggingface_hub import snapshot_download           
snapshot_download("llmware/dragon-llama-3.1-gguf", local_dir="/path/on/your/machine/", local_dir_use_symlinks=False)  

Load in your favorite GGUF inference engine, or try with llmware as follows:

from llmware.models import ModelCatalog  

# to load the model and make a basic inference
model = ModelCatalog().load_model("llmware/dragon-llama-3.1-gguf", temperature=0.0, sample=False)
response = model.inference(query, add_context=text_sample)  

Details on the prompt wrapper and other configurations are on the config.json file in the files repository.

Model Card Contact

Darren Oberst & llmware team