Downloads · 30 days
3
1% of all-time downloads
Smaug-Mixtral-v0.1-8.0bpw-exl2
bdambrosio/Smaug-Mixtral-v0.1-8.0bpw-exl2
Smaug-Mixtral-v0.1-8.0bpw-exl2 is a text generation model from bdambrosio. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
I know, I know, 6.5bpw is enough for perfection. For some people.
All-time downloads
211
Public
Repo size
46.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors46.8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mixtral
- Access
- Public
- Created
- Feb 23, 2024
- Updated
- Feb 23, 2024
- SHA
- 11e951ce
Try a prompt
Model card
From the Hugging Face model README
I know, I know, 6.5bpw is enough for perfection. For some people.
But, for those who want the best they can load, here's an 8bpw quant. Makes a difference for me, I think.
tweaked exl2 quant parameters a bit because I run 6-8k contexts:
python3 convert.py -i ../models/Smaug-Mixtral_v0.1 -o smaug_mixtral -cf smaug_mixtral -l 4096 -b 8 -hb 8 -ss 4096
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- apache-2.0
- Created
- Feb 23, 2024
- Updated
- Feb 23, 2024