Downloads · 30 days
180
0% of all-time downloads
Mixtral-8x7B-Instruct-v0.1-offloading-demo
lavawolfiee/Mixtral-8x7B-Instruct-v0.1-offloading-demo
Mixtral-8x7B-Instruct-v0.1-offloading-demo is a text generation model from lavawolfiee. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Attention quantization: HQQ 4-bit, groupsize 64, compress zero, compress scale with groupsize 256 \ Experts quantization: HQQ 2-bit, groupsize 16, compress zero, compress scale with groupsize 128
All-time downloads
55.3K
Public
Parameters
17.9B
18.2 GB on disk
Likes
27
Public
Repo makeup
Click a slice to open those files.
.safetensors18.2 GB · 100%
Parameter types
How the weights are stored.
U817.6B · 98%
Try a prompt
Model card
From the Hugging Face model README
Attention quantization: HQQ 4-bit, groupsize 64, compress zero, compress scale with groupsize 256
Experts quantization: HQQ 2-bit, groupsize 16, compress zero, compress scale with groupsize 128
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- mit
- Languages
- en, fr, it, de, es
- Created
- Dec 17, 2023
- Updated
- Dec 30, 2023