Downloads · 30 days
21
5% of all-time downloads
GPT4-X-Alpasta-30b-4bit
askmyteapot/GPT4-X-Alpasta-30b-4bit
GPT4-X-Alpasta-30b-4bit is a text generation model from askmyteapot. Use it when you need the model to write or continue text. It is set up for transformers.
Using comit <a href="https://github.com/0cc4m/GPTQ-for-LLaMa/tree/3c16fd9c7946ebe85df8d951cb742adbc1966ec7"3c16fd9</a of 0cc4m's GPTQ fork Using C4 as the calibration dataset Act-order, True-sequential, percdamp 0.1 (…
All-time downloads
395
Public
Repo size
16.9 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.safetensors16.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- May 16, 2023
- Updated
- May 17, 2023
- SHA
- bd9ae91c
Try a prompt
Model card
From the Hugging Face model README
This is a 4bit quant of https://huggingface.co/MetaIX/GPT4-X-Alpasta-30b
My secret sauce:
- Using comit <a href="https://github.com/0cc4m/GPTQ-for-LLaMa/tree/3c16fd9c7946ebe85df8d951cb742adbc1966ec7">3c16fd9</a> of 0cc4m's GPTQ fork
- Using C4 as the calibration dataset
- Act-order, True-sequential, percdamp 0.1 (<i>the default percdamp is 0.01</i>)
- No groupsize
- Will run with CUDA, does not need triton.
- Quant completed on a 'Premium GPU' and 'High Memory' Google Colab.
Benchmark results
| <b>Model<b> | <b>C4<b> | <b>WikiText2<b> | <b>PTB<b> |
|---|---|---|---|
| MetaIX's FP16 | 6.98400259 | 4.607768536 | 9.414786339 |
| This Quant | 7.292364597 | 4.954069614 | 9.754593849 |
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- May 16, 2023
- Updated
- May 17, 2023