Downloads · 30 days
29
4% of all-time downloads
gpt4-x-alpaca-13b-native-4bit-128g
Selyam/gpt4-x-alpaca-13b-native-4bit-128g
gpt4-x-alpaca-13b-native-4bit-128g is a text generation model from Selyam. Use it when you need the model to write or continue text. It is set up for transformers.
All-time downloads
690
Public
Repo size
26.2 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.pt16 GB · 61%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Apr 7, 2023
- Updated
- Apr 12, 2023
- SHA
- c2660a49
Try a prompt
Model card
From the Hugging Face model README
Update (4/1): Added ggml for Cuda model
Dataset is here (instruct): https://github.com/teknium1/GPTeacher
Okay... Two different models now. One generated in the Triton branch, one generated in Cuda. Use the Cuda one for now unless the Triton branch becomes widely used.
Cuda info (use this one): Command:
CUDA_VISIBLE_DEVICES=0 python llama.py ./models/chavinlo-gpt4-x-alpaca --wbits 4 --true-sequential --groupsize 128 --save gpt-x-alpaca-13b-native-4bit-128g-cuda.pt
Prev. info
Quantized on GPTQ-for-LLaMa commit 5955e9c67d9bfe8a8144ffbe853c2769f1e87cdd
GPTQ 4bit quantization of: https://huggingface.co/chavinlo/gpt4-x-alpaca
Note: This was quantized with this branch of GPTQ-for-LLaMA: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
Because of this, it appears to be incompatible with Oobabooga at the moment. Stay tuned?
Command:
CUDA_VISIBLE_DEVICES=0 python llama.py ./models/chavinlo-gpt4-x-alpaca --wbits 4 --true-sequential --act-order --groupsize 128 --save gpt-x-alpaca-13b-native-4bit-128g.pt
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Apr 7, 2023
- Updated
- Apr 12, 2023