Downloads · 30 days
19
6% of all-time downloads
SUS-Chat-34B_2bit
Minami-su/SUS-Chat-34B_2bit
SUS-Chat-34B_2bit is a text generation model from Minami-su. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
You can run it on 11G mem GPU,quantize base QuIP method, a weights-only quantization method that is able to achieve near fp16 performance using only 2 bits per weight.
All-time downloads
309
Public
Parameters
5.1B
10.2 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors10.2 GB · 100%
Parameter types
How the weights are stored.
I164.2B · 82%
Try a prompt
Model card
From the Hugging Face model README
You can run it on 11G mem GPU,quantize base QuIP# method, a weights-only quantization method that is able to achieve near fp16 performance using only 2 bits per weight.
url:https://github.com/Cornell-RelaxML/quip-sharp/tree/release20231203
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Created
- Dec 22, 2023
- Updated
- Dec 31, 2023