Meta-Llama-3-8B-Instruct-GGUF

Meta-Llama-3-8B-Instruct GGUF

Original model: Meta-Llama-3-8B-Instruct

Model creator: Meta

Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, we took great care to optimize helpfulness and safety.

Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

This repo contains GGUF format model files for Meta’s Llama-3-8B-Instruct, updated as of 2024-04-29 to incorporate tokenization improvements, as well as previous interventions to handle the <|eot_id|> special token as EOS token.

Learn more on Meta’s Llama 3 page.

What is GGUF?

GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. Converted with llama.cpp build 2763 (revision ffe666), using autogguf.

Prompt template

<|start_header_id|>system<|end_header_id|>

{{system_prompt}}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{prompt}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

---

Download & run with cnvrs on iPhone, iPad, and Mac!

cnvrs is the best app for private, local AI on your device:

• create & save Characters with custom system prompts & temperature settings
• download and experiment with any GGUF model you can find on HuggingFace!
• make it your own with custom Theme colors
• powered by Metal ⚡️ & Llama.cpp, with haptics during response streaming!
• try it out yourself today, on Testflight!
• follow cnvrs on twitter to stay up to date

---

Original Model Evaluation

<table> <tr> <td><strong>Benchmark</strong> </td> <td><strong>Llama 3 8B</strong> </td> <td><strong>Llama 2 7B</strong> </td> <td><strong>Llama 2 13B</strong> </td> </tr> <tr> <td>MMLU (5-shot) </td> <td><b>68.4</b> </td> <td>34.1 </td> <td>47.8 </td> </tr> <tr> <td>GPQA (0-shot) </td> <td><b>34.2</b> </td> <td>21.7 </td> <td>22.3 </td> </tr> <tr> <td>HumanEval (0-shot) </td> <td><b>62.2</b> </td> <td>7.9 </td> <td>14.0 </td> </tr> <tr> <td>GSM-8K (8-shot, CoT) </td> <td><b>79.6</b> </td> <td>25.7 </td> <td>77.4 </td> </tr> <tr> <td>MATH (4-shot, CoT) </td> <td><b>30.0</b> </td> <td>3.8 </td> <td>6.7 </td> </tr> </table>