Downloads · 30 days
158
12% of all-time downloads
GPT4-x-AlpacaDente2-30b-4bit
askmyteapot/GPT4-x-AlpacaDente2-30b-4bit
GPT4-x-AlpacaDente2-30b-4bit is a text generation model from askmyteapot. Use it when you need the model to write or continue text. It is set up for transformers.
Using comit <a href="https://github.com/0cc4m/GPTQ-for-LLaMa/tree/3c16fd9c7946ebe85df8d951cb742adbc1966ec7"3c16fd9</a of 0cc4m's GPTQ fork Using PTB as the calibration dataset Act-order, True-sequential, percdamp 0.1…
All-time downloads
1.3K
Public
Repo size
16.9 GB
Likes
10
Public
Repo makeup
Click a slice to open those files.
.safetensors16.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- May 5, 2023
- Updated
- May 5, 2023
- SHA
- 99142824
Try a prompt
Model card
From the Hugging Face model README
This is a 4bit quant of https://huggingface.co/Aeala/GPT4-x-AlpacaDente2-30b
My secret sauce:
- Using comit <a href="https://github.com/0cc4m/GPTQ-for-LLaMa/tree/3c16fd9c7946ebe85df8d951cb742adbc1966ec7">3c16fd9</a> of 0cc4m's GPTQ fork
- Using PTB as the calibration dataset
- Act-order, True-sequential, percdamp 0.1 (<i>the default percdamp is 0.01</i>)
- No groupsize
- Will run with CUDA, does not need triton.
- Quant completed on a 'Premium GPU' and 'High Memory' Google Colab.
Benchmark results
| <b>Model<b> | <b>C4<b> | <b>WikiText2<b> | <b>PTB<b> |
|---|---|---|---|
| Aeala's FP16 | 7.05504846572876 | 4.662261962890625 | 24.547462463378906 |
| This Quant | 7.326207160949707 | 4.957101345062256 | 24.941526412963867 |
| Aeala's Quant <a href="https://huggingface.co/Aeala/GPT4-x-AlpacaDente2-30b/resolve/main/4bit.safetensors">here</a> | 7.332120418548584 | 5.016242980957031 | 25.576189041137695 |
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- May 5, 2023
- Updated
- May 5, 2023