Downloads · 30 days
15
5% of all-time downloads
ChenMnZ/Mixtral-8x7B-v0.1-OmniQuantv2-w4a16g128
Mixtral-8x7B-v0.1-OmniQuantv2-w4a16g128 is a text generation model from ChenMnZ. Use it when you need the model to write or continue text. It is set up for transformers.
Downloads · 30 days
15
5% of all-time downloads
All-time downloads
289
Public
Parameters
6.5B
24.7 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors24.7 GB · 100%
How the weights are stored.
I325.9B · 90%
From the Hugging Face model README
Paper: https://arxiv.org/abs/2308.13137
Code: https://github.com/OpenGVLab/OmniQuant
To run this model, refer https://github.com/OpenGVLab/OmniQuant/blob/main/runing_quantized_mixtral_7bx8.ipynb for more details.