Downloads · 30 days
11
4% of all-time downloads
Mixtral-8x7B-Instruct-v0.1-offloading-demo
inventbot/Mixtral-8x7B-Instruct-v0.1-offloading-demo
Mixtral-8x7B-Instruct-v0.1-offloading-demo is a text generation model from inventbot. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Attention quantization: HQQ 4-bit, groupsize 64, compress zero, compress scale with groupsize 256 \ Experts quantization: HQQ 2-bit, groupsize 16, compress zero, compress scale with groupsize 128
All-time downloads
260
Public
Parameters
17.9B
18.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors18.2 GB · 100%
Parameter types
How the weights are stored.
U817.6B · 98%
Try a prompt
Model card
From the Hugging Face model README
Attention quantization: HQQ 4-bit, groupsize 64, compress zero, compress scale with groupsize 256
Experts quantization: HQQ 2-bit, groupsize 16, compress zero, compress scale with groupsize 128
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- mit
- Languages
- en, fr, it, de, es
- Created
- Feb 17, 2024
- Updated
- May 26, 2024