Downloads · 30 days
281
1% of all-time downloads
gpt4-x-alpaca-13b-native-4bit-128g-cuda
4bit/gpt4-x-alpaca-13b-native-4bit-128g-cuda
gpt4-x-alpaca-13b-native-4bit-128g-cuda is a text generation model from 4bit. Use it when you need the model to write or continue text. It is set up for transformers.
Quant of https://huggingface.co/chavinlo/gpt4-x-alpaca
All-time downloads
32.3K
Public
Repo size
8.1 GB
Likes
12
Public
Repo makeup
Click a slice to open those files.
.safetensors8.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Apr 9, 2023
- Updated
- Apr 9, 2023
- SHA
- 1bd77a5a
Try a prompt
Model card
From the Hugging Face model README
Quant of https://huggingface.co/chavinlo/gpt4-x-alpaca
There's already one located at https://huggingface.co/anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g, but neither the triton nor cuda version they uploaded seem to want to work on older versions of GPTQ-for-LLaMA such as the one currently used with KoboldAI for 4bit support on 0cc4m's fork.
This was quantized with cuda, not triton.
python llama.py ./gpt4-x-alpaca c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors gpt-x-alpaca-13b-native-4bit-128g-cuda.safetensors
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Apr 9, 2023
- Updated
- Apr 9, 2023