Downloads · 30 days
63
3% of all-time downloads
Meta-Llama-3.1-8B-Instruct-GGUF
mukel/Meta-Llama-3.1-8B-Instruct-GGUF
Meta-Llama-3.1-8B-Instruct-GGUF is a machine learning model from mukel. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.1.
Pure .gguf Q40 and Q80 quantizations of Llama 3 8B instruct, ready to consume by llama3.java.
All-time downloads
1.8K
Public
Repo size
13.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf13.1 GB · 100%
At a glance
- License
- llama3.1
- Access
- Public
- Created
- Jul 26, 2024
- Updated
- Jul 26, 2024
- SHA
- 0d1ee3a9
Model card
From the Hugging Face model README
GGUF models for llama3.java
Pure .gguf Q4_0 and Q8_0 quantizations of Llama 3 8B instruct, ready to consume by llama3.java.
In the wild, Q8_0 quantizations are fine, but Q4_0 quantizations are rarely pure e.g. the output.weights tensor is quantized with Q6_K, instead of Q4_0.
A pure Q4_0 quantization can be generated from a high precision (F32, F16, BFLOAT16) .gguf source with the quantize utility from llama.cpp as follows:
./quantize --pure ./Meta-Llama-3-8B-Instruct-F32.gguf ./Meta-Llama-3-8B-Instruct-Q4_0.gguf Q4_0
Meta-Llama-3.1-8B-Instruct-GGUF
- This is GGUF quantized version of meta-llama/Meta-Llama-3.1-8B-Instruct created using llama.cpp
- License
- llama3.1
- Created
- Jul 26, 2024
- Updated
- Jul 26, 2024