Downloads · 30 days
0
Llama-3_1-8B-Q40-Instruct-Distributed-Llama
b4rtaz/Llama-3_1-8B-Q40-Instruct-Distributed-Llama
Llama-3_1-8B-Q40-Instruct-Distributed-Llama is a text generation model from b4rtaz. Use it when you need the model to write or continue text. It is set up for distributed-llama. The card lists the license as llama3.1.
This is converted Llama 3.1 8B Instruct model to Distributed Llama format. The model is quantized to Q40.
Access
Public
Updated Feb 25, 2025
Repo size
6.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.m6.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- distributed-llama
- License
- llama3.1
- Access
- Public
- Created
- Jul 25, 2024
- Updated
- Feb 25, 2025
- SHA
- c4b7e78a
Try a prompt
Model card
From the Hugging Face model README
This is converted Llama 3.1 8B Instruct model to Distributed Llama format. The model is quantized to Q40.
🚀 How to Run?
- ⏬ Download this repository.
- ⏬ Download Distributed Llama repository.
- 🔨 Build Distributed Llama:
make dllama
- 🚀 Run Distributed Llama:
./dllama chat --model dllama_model_llama3.1_instruct_q40.m --tokenizer dllama_tokenizer_llama_3_1.t --buffer-float-type q80 --nthreads 4
🎩 License
- Task
- Text Generation
- Library
- distributed-llama
- License
- llama3.1
- Created
- Jul 25, 2024
- Updated
- Feb 25, 2025