Downloads · 30 days
0
Llama-3_2-3B-Q40-Instruct-Distributed-Llama
b4rtaz/Llama-3_2-3B-Q40-Instruct-Distributed-Llama
Llama-3_2-3B-Q40-Instruct-Distributed-Llama is a text generation model from b4rtaz. Use it when you need the model to write or continue text. It is set up for distributed-llama. The card lists the license as llama3.2.
This is converted Llama 3.2 3B Instruct model to Distributed Llama format. The model is quantized to Q40.
Access
Public
Updated Feb 25, 2025
Repo size
8.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.m3.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- distributed-llama
- License
- llama3.2
- Access
- Public
- Created
- Oct 13, 2024
- Updated
- Feb 25, 2025
- SHA
- 35acc6dd
Try a prompt
Model card
From the Hugging Face model README
This is converted Llama 3.2 3B Instruct model to Distributed Llama format. The model is quantized to Q40.
🚀 How to Run?
- ⏬ Download this repository.
- ⏬ Download Distributed Llama repository.
- 🔨 Build Distributed Llama:
make dllama
- 🚀 Run Distributed Llama:
./dllama chat --model dllama_model_llama3_2_3b_instruct_q40.m --tokenizer dllama_tokenizer_llama3_2_3b_instruct_q40.t --buffer-float-type q80 --nthreads 4 --max-seq-len 8192
🎩 License
- Task
- Text Generation
- Library
- distributed-llama
- License
- llama3.2
- Created
- Oct 13, 2024
- Updated
- Feb 25, 2025