Downloads · 30 days
15
4% of all-time downloads
Mistral-Nemo-Instruct-2407-AQLM-PV-2Bit-1x16-hf
ISTA-DASLab/Mistral-Nemo-Instruct-2407-AQLM-PV-2Bit-1x16-hf
Mistral-Nemo-Instruct-2407-AQLM-PV-2Bit-1x16-hf is a machine learning model from ISTA-DASLab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Official AQLM quantization of mistralai/Mistral-Nemo-Instruct-2407 finetuned with PV-Tuning.
All-time downloads
403
Public
Parameters
2.9B
5.7 GB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors5.7 GB · 100%
Parameter types
How the weights are stored.
F161.5B · 52%
Model card
From the Hugging Face model README
Official AQLM quantization of mistralai/Mistral-Nemo-Instruct-2407 finetuned with PV-Tuning.
For this quantization, we used 1 codebook of 16 bits and groupsize of 8.
Results:
| Model | Quantization | MMLU (5-shot) | ArcC | ArcE | Hellaswag | PiQA | Winogrande | Model size, Gb |
|---|---|---|---|---|---|---|---|---|
| mistralai/Mistral-Nemo-Instruct-2407 | None | 0.6819 | 0.5606 | 0.8241 | 0.6332 | 0.8090 | 0.7498 | 24.5 |
| 1x16g8 | 0.6071 | 0.5017 | 0.7942 | 0.5930 | 0.7987 | 0.7356 | 5.8 |
Note
We used lm-eval=0.4.0 for evaluation.
- Type
- mistral
- License
- apache-2.0
- Languages
- en, fr, de, es, it, pt
- Created
- Aug 16, 2024
- Updated
- Aug 16, 2024