Downloads · 30 days
12
5% of all-time downloads
Wizard-Vicuna-30B-SuperHOT-8k-test-GPTQ
QMB15/Wizard-Vicuna-30B-SuperHOT-8k-test-GPTQ
Wizard-Vicuna-30B-SuperHOT-8k-test-GPTQ is a text generation model from QMB15. Use it when you need the model to write or continue text. It is set up for transformers.
This is https://huggingface.co/ehartford/Wizard-Vicuna-30B-Uncensored, merged with https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test, then quantized to 4bit with AutoGPTQ. There are two quantized versions…
All-time downloads
244
Public
Repo size
34.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors34.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Jun 26, 2023
- Updated
- Jun 27, 2023
- SHA
- 2a5a8211
Try a prompt
Model card
From the Hugging Face model README
This is https://huggingface.co/ehartford/Wizard-Vicuna-30B-Uncensored, merged with https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test, then quantized to 4bit with AutoGPTQ. There are two quantized versions. One is a plain 4bit version with only act-order and no groupsize. The other is an experimental version using groupsize 128, act-order, and kaiokendev's ScaledLLamaAttention monkey patch applied during quantization, the idea being to help the calibration account for the new scale. It seems to have worked as it improves by around 0.04 ppl vs the unpatched quant - maybe not worth the trouble, but it's better so I'll put it up anyway.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Jun 26, 2023
- Updated
- Jun 27, 2023