Downloads · 30 days
12
3% of all-time downloads
airoboros-13b-gpt4-TRITON
BigStationW/airoboros-13b-gpt4-TRITON
airoboros-13b-gpt4-TRITON is a text generation model from BigStationW. Use it when you need the model to write or continue text. It is set up for transformers.
This is the gptq 4bit quantization of this model: https://huggingface.co/jondurbin/airoboros-13b-gpt4
All-time downloads
367
Public
Repo size
7.3 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors7.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Jun 4, 2023
- Updated
- Jun 5, 2023
- SHA
- 92f2702d
Try a prompt
Model card
From the Hugging Face model README
This is the gptq 4bit quantization of this model: https://huggingface.co/jondurbin/airoboros-13b-gpt4
This quantization was made by using this repository: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
And I used the triton branch with all the gptq implementations available (true_sequential + act_order + groupsize 128)
CUDA_VISIBLE_DEVICES=0 python llama.py ./airoboros-13b-gpt4-TRITON c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors airoboros-13b-gpt4-128g-ts-ao.safetensors
Airoboros 13b gpt4 TRITON (g128 - ts - ao)
PPL: 5.480927467346191
max memory(MiB): 8590.25
Airoboros 13b gpt4 CUDA (g128 - ts)
PPL: 5.535770893096924
max memory(MiB): 8750.4697265625
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Jun 4, 2023
- Updated
- Jun 5, 2023