Downloads · 30 days
12
4% of all-time downloads
Mixtral-8x7B-Instruct-v0.1-offloading-hqq-4bit-3bit
lavawolfiee/Mixtral-8x7B-Instruct-v0.1-offloading-hqq-4bit-3bit
Mixtral-8x7B-Instruct-v0.1-offloading-hqq-4bit-3bit is a text generation model from lavawolfiee. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Attention quantization: HQQ 4-bit, groupsize 64, compress zero, compress scale with groupsize 256 \ Experts quantization: HQQ 3-bit, groupsize 64, compress zero, compress scale with groupsize 128
All-time downloads
327
Public
Parameters
6.9B
20.7 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors20.7 GB · 100%
Parameter types
How the weights are stored.
I324.5B · 65%
Try a prompt
Model card
From the Hugging Face model README
Attention quantization: HQQ 4-bit, groupsize 64, compress zero, compress scale with groupsize 256
Experts quantization: HQQ 3-bit, groupsize 64, compress zero, compress scale with groupsize 128
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- mit
- Languages
- en, fr, it, de, es
- Created
- Dec 26, 2023
- Updated
- Dec 30, 2023