Downloads · 30 days
36
7% of all-time downloads
llama13b-4bit-v2
sardukar/llama13b-4bit-v2
llama13b-4bit-v2 is a text generation model from sardukar. Use it when you need the model to write or continue text. It is set up for transformers.
Quantized Meta AI's LLaMA in 4bit with the help of GPTQ algorithm v2.
All-time downloads
546
Public
Repo size
14.3 GB
Likes
5
Public
Repo makeup
Click a slice to open those files.
.safetensors14.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Mar 26, 2023
- Updated
- Apr 8, 2023
- SHA
- 84094fd8
Try a prompt
Model card
From the Hugging Face model README
Quantized Meta AI's LLaMA in 4bit with the help of GPTQ algorithm v2.
- llama13b-4bit-ts-ao-g128-v2.safetensors GPTQ implementation - https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/49efe0b67db4b40eac2ae963819ebc055da64074
Conversion process:
CUDA_VISIBLE_DEVICES=0 python llama.py ./llama-13b c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors ./q4/llama13b-4bit-ts-ao-g128-v2.safetensors
- llama13b-4bit-v2.safetensors GPTQ implementation - https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/841feedde876785bc8022ca48fd9c3ff626587e2
Note: This model will fail to load with current GPTQ-for-LLaMa implementation
Conversion process
CUDA_VISIBLE_DEVICES=0 python llama.py ./llama-13b c4 --wbits 4 --true-sequential --act-order --save_safetensors ./q4/llama13b-4bit-v2.safetensors
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Mar 26, 2023
- Updated
- Apr 8, 2023