Downloads · 30 days
18
3% of all-time downloads
jais-13b-chat-GPTQ-4bits
haouarin/jais-13b-chat-GPTQ-4bits
jais-13b-chat-GPTQ-4bits is a text generation model from haouarin. Use it when you need the model to write or continue text. It is set up for transformers.
This is the quantized version 4-bits created using autotrain, but it doesn't work.
All-time downloads
581
Public
Repo size
7.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin7.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- jais
- Access
- Public
- Created
- Sep 3, 2023
- Updated
- Sep 6, 2023
- SHA
- ecd905c3
Try a prompt
Model card
From the Hugging Face model README
This is the quantized version 4-bits created using autotrain, but it doesn't work.
Error
GPU
CPU
Quantization Process
!pip install auto-gptq
!pip install git+https://github.com/huggingface/optimum.git
!pip install git+https://github.com/huggingface/transformers.git
!pip install --upgrade accelerate
from transformers import AutoModelForCausalLM, AutoTokenizer,GPTQConfig
tokenizer = AutoTokenizer.from_pretrained("inception-mbzuai/jais-13b-chat")
gptq_config = GPTQConfig(bits=4, dataset = "c4", tokenizer=tokenizer)
model = AutoModelForCausalLM.from_pretrained('inception-mbzuai/jais-13b-chat', quantization_config=gptq_config,trust_remote_code=True)
- Task
- Text Generation
- Library
- transformers
- Type
- jais
- Created
- Sep 3, 2023
- Updated
- Sep 6, 2023