Downloads · 30 days
13
4% of all-time downloads
Wizard-Vicuna-13B-Uncensored-SpQR
Honkware/Wizard-Vicuna-13B-Uncensored-SpQR
Wizard-Vicuna-13B-Uncensored-SpQR is a text generation model from Honkware. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This model is an SpQR 4-bit quantization of the original Wizard-Vicuna-13B-Uncensored-HF
All-time downloads
324
Public
Repo size
26 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors26 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- llama
- Access
- Public
- Created
- Jul 9, 2023
- Updated
- Jul 10, 2023
- SHA
- 5df3c71d
Try a prompt
Model card
From the Hugging Face model README
Wizard-Vicuna-13B-Uncensored-SpQR
Overview
This model is an SpQR 4-bit quantization of the original Wizard-Vicuna-13B-Uncensored-HF
Quantization Specifications
- Quantization: 4-bit, group size of 16, per-channel with scale and zero-point of 3 bits.
- Outliers: Threshold set at 0.2.
- Permutation Order:
act_order. - Dampening: Set at 1e0.
- Sampling: 128 samples.
- Logging: Via Weights & Biases.
Evaluation Metrics
The following perplexity scores were obtained on various datasets:
| Dataset | Perplexity |
|---|---|
| c4 | 7.354 |
| wikitext2 | 5.685 |
| ptb | 20.822 |
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- other
- Created
- Jul 9, 2023
- Updated
- Jul 10, 2023