Downloads · 30 days
0
opt-6b-4-bit
autobots/opt-6b-4-bit
opt-6b-4-bit is a machine learning model from autobots. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Quantized for the older GPTQ before it broke all the models.
Access
Public
Updated Mar 26, 2023
Repo size
10.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt10.5 GB · 100%
At a glance
- License
- other
- Access
- Public
- Created
- Mar 26, 2023
- Updated
- Mar 26, 2023
- SHA
- 6a5820d6
Model card
From the Hugging Face model README
Quantized for the older GPTQ before it broke all the models.
Use with
https://github.com/Ph0rk0z/text-generation-webui-testing
https://github.com/Ph0rk0z/GPTQ-Merged
https://github.com/Curlypla/peft-GPTQ
Clone the 2 repos into text-generation-webui-testing/repositories
python cuda_setup.py install inside GPTQ-Merged to compile nvidia kernel.
python server.py --cai-chat --gptq-bits 4 --model opt-6b --autograd
Don't forget to get configs from: https://huggingface.co/facebook/opt-6.7b/tree/main
You only need the json files, don't forget merges.txt
- License
- other
- Created
- Mar 26, 2023
- Updated
- Mar 26, 2023