Downloads · 30 days
22
3% of all-time downloads
Llama-2-13b-AQLM-2Bit-2x8-hf
ISTA-DASLab/Llama-2-13b-AQLM-2Bit-2x8-hf
Llama-2-13b-AQLM-2Bit-2x8-hf is a text generation model from ISTA-DASLab. Use it when you need the model to write or continue text. It is set up for transformers.
Official AQLM quantization of meta-llama/Llama-2-13b-hf.
All-time downloads
735
Public
Parameters
3.5B
3.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.8 GB · 100%
Parameter types
How the weights are stored.
I83.2B · 91%
Try a prompt
Model card
From the Hugging Face model README
Official AQLM quantization of meta-llama/Llama-2-13b-hf.
For this quantization, we used 2 codebook of 8 bits.
Selected evaluation results for this and other models:
| Model | Quantization | WikiText 2 PPL | Model size, Gb | |
|---|---|---|---|---|
| Llama-2-13b | - | 4.57 | 26.0 | |
| 2x8 | 5.63 | 3.8 |
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Jun 4, 2024
- Updated
- Jun 4, 2024