Downloads · 30 days
13
5% of all-time downloads
gpt2-medium-AutoGPTQ-4bit-128g
smpanaro/gpt2-medium-AutoGPTQ-4bit-128g
gpt2-medium-AutoGPTQ-4bit-128g is a text generation model from smpanaro. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
gpt2-medium quantized to 4-bit using AutoGPTQ.
All-time downloads
280
Public
Repo size
366 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors366 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Feb 28, 2024
- Updated
- Feb 28, 2024
- SHA
- c86ffce7
Try a prompt
Datasets
Model card
From the Hugging Face model README
gpt2-medium quantized to 4-bit using AutoGPTQ.
To use, first install AutoGPTQ:
pip install auto-gptq
Then load the model from the hub:
from transformers import AutoModelForCausalLM, AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
model_name = "smpanaro/gpt2-medium-AutoGPTQ-4bit-128g"
model = AutoGPTQForCausalLM.from_quantized(model_name)
| Model | 4-Bit Perplexity | 16-Bit Perplexity | Delta |
|---|---|---|---|
| smpanaro/gpt2-AutoGPTQ-4bit-128g | 26.5000 | 25.1875 | 1.3125 |
| smpanaro/gpt2-medium-AutoGPTQ-4bit-128g | 19.1719 | 18.4739 | 0.698 |
| smpanaro/gpt2-large-AutoGPTQ-4bit-128g | 16.6875 | 16.4541 | 0.2334 |
| smpanaro/gpt2-xl-AutoGPTQ-4bit-128g | 14.9297 | 14.7951 | 0.1346 |
| <sub>Wikitext perplexity measured as in the huggingface docs, lower is better</sub> |
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Created
- Feb 28, 2024
- Updated
- Feb 28, 2024