Downloads · 30 days
19
6% of all-time downloads
gpt2-AutoGPTQ-4bit-128g
smpanaro/gpt2-AutoGPTQ-4bit-128g
gpt2-AutoGPTQ-4bit-128g is a text generation model from smpanaro. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
All-time downloads
301
Public
Repo size
201 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors201 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Feb 28, 2024
- Updated
- Feb 28, 2024
- SHA
- 4ad090aa
Try a prompt
Datasets
Model card
From the Hugging Face model README
gpt2 quantized to 4-bit using AutoGPTQ.
To use, first install AutoGPTQ:
pip install auto-gptq
Then load the model from the hub:
from transformers import AutoModelForCausalLM, AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
model_name = "smpanaro/gpt2-AutoGPTQ-4bit-128g"
model = AutoGPTQForCausalLM.from_quantized(model_name)
| Model | 4-Bit Perplexity | 16-Bit Perplexity | Delta |
|---|---|---|---|
| smpanaro/gpt2-AutoGPTQ-4bit-128g | 26.5000 | 25.1875 | 1.3125 |
| smpanaro/gpt2-medium-AutoGPTQ-4bit-128g | 19.1719 | 18.4739 | 0.698 |
| smpanaro/gpt2-large-AutoGPTQ-4bit-128g | 16.6875 | 16.4541 | 0.2334 |
| smpanaro/gpt2-xl-AutoGPTQ-4bit-128g | 14.9297 | 14.7951 | 0.1346 |
| <sub>Wikitext perplexity measured as in the huggingface docs, lower is better</sub> |
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Created
- Feb 28, 2024
- Updated
- Feb 28, 2024