Downloads · 30 days
10
2% of all-time downloads
stable-vicuna-13B-4bit-128g-cuda
tsumeone/stable-vicuna-13B-4bit-128g-cuda
stable-vicuna-13B-4bit-128g-cuda is a text generation model from tsumeone. Use it when you need the model to write or continue text. It is set up for transformers.
Quantized version of this: https://huggingface.co/TheBloke/stable-vicuna-13B-HF
All-time downloads
480
Public
Repo size
7.5 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors7.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Apr 30, 2023
- Updated
- Apr 30, 2023
- SHA
- df38478e
Try a prompt
Model card
From the Hugging Face model README
Quantized version of this: https://huggingface.co/TheBloke/stable-vicuna-13B-HF
Big thank you to TheBloke for uploading the HF version above. Unfortunately, his GPTQ quant doesn't run on 0cc4m's fork of KAI/GPTQ so I am uploading one that does.
GPTQ quantization using https://github.com/0cc4m/GPTQ-for-LLaMa for compatibility with 0cc4m's fork of KoboldAI.
Command used to quantize:
python llama.py c:\stable-vicuna-13B-HF c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors 4bit-128g.safetensors
This model works best with the following prompting. Also, it really does not like to stop on its own and will likely keep going on forever if you let it.
### Human:
What is 2+2?
### Assistant:
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Apr 30, 2023
- Updated
- Apr 30, 2023