Downloads · 30 days
13
7% of all-time downloads
gpt2-large-AutoGPTQ-4bit-128g
smpanaro/gpt2-large-AutoGPTQ-4bit-128g
gpt2-large-AutoGPTQ-4bit-128g is a text generation model from smpanaro. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
gpt2-large quantized to 4-bit using AutoGPTQ.
All-time downloads
199
Public
Repo size
630 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors630 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Feb 28, 2024
- Updated
- Feb 28, 2024
- SHA
- aa5636f7
Try a prompt
Datasets
Model card
From the Hugging Face model README
gpt2-large quantized to 4-bit using AutoGPTQ.
To use, first install AutoGPTQ:
pip install auto-gptq
Then load the model from the hub:
from transformers import AutoModelForCausalLM, AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
model_name = "smpanaro/gpt2-large-AutoGPTQ-4bit-128g"
model = AutoGPTQForCausalLM.from_quantized(model_name)
| Model | 4-Bit Perplexity | 16-Bit Perplexity | Delta |
|---|---|---|---|
| smpanaro/gpt2-AutoGPTQ-4bit-128g | 26.5000 | 25.1875 | 1.3125 |
| smpanaro/gpt2-medium-AutoGPTQ-4bit-128g | 19.1719 | 18.4739 | 0.698 |
| smpanaro/gpt2-large-AutoGPTQ-4bit-128g | 16.6875 | 16.4541 | 0.2334 |
| smpanaro/gpt2-xl-AutoGPTQ-4bit-128g | 14.9297 | 14.7951 | 0.1346 |
| <sub>Wikitext perplexity measured as in the huggingface docs, lower is better</sub> |
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Created
- Feb 28, 2024
- Updated
- Feb 28, 2024