4bit

llama-13b-4bit-gr128

4bit/llama-13b-4bit-gr128

llama-13b-4bit-gr128 is a text generation model from 4bit. Use it when you need the model to write or continue text. It is set up for transformers.

Generated with: --wbits 4 --groupsize 128 --true-sequential --new-eval --faster-kernel