Downloads · 30 days
10
2% of all-time downloads
Llama-3.1-8B-Q8R16-GGUF
canes33/Llama-3.1-8B-Q8R16-GGUF
Llama-3.1-8B-Q8R16-GGUF is a machine learning model from canes33. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a quantized version of meta-llama/Llama-3.1-8B using Q8R16 quantization method via llama.cpp.
All-time downloads
455
Public
Repo size
16.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf8.1 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Nov 24, 2025
- Updated
- Nov 26, 2025
- SHA
- fea4b80f
Base models
Model card
From the Hugging Face model README
Llama-3.1-8B-Q8R16-GGUF
This is a quantized version of meta-llama/Llama-3.1-8B using Q8R16 quantization method via llama.cpp.
Quantization Details
- Method: Q8R16
- Format: GGUF
- Base Model: meta-llama/Llama-3.1-8B
- Quantized using: llama.cpp llama-quantize binary
Usage
Using llama.cpp
# Download the model
wget https://huggingface.co/canes33/Llama-3.1-8B-Q8R16-GGUF/resolve/main/model-Q8R16.gguf
# Run inference
./llama-cli -m model-Q8R16.gguf -p "Your prompt here"
Using Python (llama-cpp-python)
from llama_cpp import Llama
llm = Llama(model_path="model-Q8R16.gguf")
output = llm("Your prompt here", max_tokens=100)
print(output)
Using other tools
This GGUF model is compatible with:
Model File Info
- Filename: model-Q8R16.gguf
- Quantization: Q8R16
- License
- apache-2.0
- Created
- Nov 24, 2025
- Updated
- Nov 26, 2025