Downloads · 30 days
13
3% of all-time downloads
llama-13b-SuperCOT-4bit-TRITON
BigStationW/llama-13b-SuperCOT-4bit-TRITON
llama-13b-SuperCOT-4bit-TRITON is a text generation model from BigStationW. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This is the gptq 4bit quantization of this model: https://huggingface.co/kaiokendev/SuperCOT-LoRA
All-time downloads
450
Public
Repo size
7.3 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.safetensors7.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- llama
- Access
- Public
- Created
- Apr 23, 2023
- Updated
- Apr 23, 2023
- SHA
- 83c9e955
Try a prompt
Model card
From the Hugging Face model README
This is the gptq 4bit quantization of this model: https://huggingface.co/kaiokendev/SuperCOT-LoRA
This quantization was made by using this repository: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
And I used the triton branch with all the gptq implementations available (true_sequential + act_order + groupsize 128) CUDA_VISIBLE_DEVICES=0 python llama.py ./llama-13b-SuperCOT-4bit-TRITON c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors llama-13b-SuperCOT-4bit-TRITON.safetensors
To use the triton model on oobabooga's webui, you can refer to this post to get rid of all the errors you can encounter: https://github.com/oobabooga/text-generation-webui/issues/734
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- other
- Created
- Apr 23, 2023
- Updated
- Apr 23, 2023