Downloads · 30 days
154
1% of all-time downloads
MarkP1929/oasst-llama-13b-2-epochs-GPTQ-4bit-128g
oasst-llama-13b-2-epochs-GPTQ-4bit-128g is a text generation model from MarkP1929. Use it when you need the model to write or continue text. It is set up for transformers.
This is a quantised version in safetensor format of the oasst-llama-13b-2-epochs model from dvruette/oasst-llama-13b-2-epochs
Downloads · 30 days
154
1% of all-time downloads
All-time downloads
28.1K
Public
Repo size
7.3 GB
Likes
3
Public
Click a slice to open those files.
.safetensors7.3 GB · 100%
From the Hugging Face model README
This is a quantised version in safetensor format of the oasst-llama-13b-2-epochs model from dvruette/oasst-llama-13b-2-epochs
It has a siginficant speed up for inference when used on oobabooga.
Run with.. python server.py --model oasst-llama-13b-2-epochs-GPTQ-4bit-128g --wbits 4 --groupsize 128