Downloads · 30 days
40
5% of all-time downloads
autoj-13b-GPTQ-4bits
GAIR/autoj-13b-GPTQ-4bits
autoj-13b-GPTQ-4bits is a text generation model from GAIR. Use it when you need the model to write or continue text. It is set up for transformers.
This is a GPTQ 4bits version of Auto-J-13B. We convert it using this script (by TheBroke).
All-time downloads
817
Public
Parameters
13B
7.9 GB on disk
Likes
4
Public
Repo makeup
Click a slice to open those files.
.safetensors7.9 GB · 100%
Parameter types
How the weights are stored.
I3212.7B · 97%
Try a prompt
Model card
From the Hugging Face model README
This is a GPTQ 4bits version of Auto-J-13B. We convert it using this script (by TheBroke).
To use the 4bits version of Auto-J, you need to install the following packages:
pip install safetensors
pip install transformers>=4.32.0 optimum>=1.12.0
pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/ # Use cu117 if on CUDA 11.7
It takes about 8GB VRAM to load this model, and we provide an example for using it in example_gptq4bits.py.
Note that the behaviours of the quantized model and the original one might be different.
Please refer to our github repo for more datails.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Languages
- en
- Created
- Oct 22, 2023
- Updated
- Oct 22, 2023