Downloads · 30 days
25
5% of all-time downloads
llama7b-4bit-v2
sardukar/llama7b-4bit-v2
llama7b-4bit-v2 is a text generation model from sardukar. Use it when you need the model to write or continue text. It is set up for transformers.
Quantized Meta AI's LLaMA in 4bit with the help of GPTQ algorithm v2. GPTQ implementation - https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/49efe0b67db4b40eac2ae963819ebc055da64074
All-time downloads
523
Public
Repo size
3.9 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors3.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Apr 6, 2023
- Updated
- Apr 6, 2023
- SHA
- c024d474
Try a prompt
Model card
From the Hugging Face model README
Quantized Meta AI's LLaMA in 4bit with the help of GPTQ algorithm v2. GPTQ implementation - https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/49efe0b67db4b40eac2ae963819ebc055da64074
Conversion process
CUDA_VISIBLE_DEVICES=0 python llama.py ./llama-7b c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors ./q4/llama7b-4bit-ts-ao-g128-v2.safetensors
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Apr 6, 2023
- Updated
- Apr 6, 2023