Downloads · 30 days
12
3% of all-time downloads
Manticore-13b-GPTQ-Triton
BigStationW/Manticore-13b-GPTQ-Triton
Manticore-13b-GPTQ-Triton is a text generation model from BigStationW. Use it when you need the model to write or continue text. It is set up for transformers.
This is the gptq 4bit quantization of this model: https://huggingface.co/openaccess-ai-collective/manticore-13b
All-time downloads
367
Public
Repo size
7.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors7.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- May 21, 2023
- Updated
- May 21, 2023
- SHA
- b82e6415
Try a prompt
Model card
From the Hugging Face model README
This is the gptq 4bit quantization of this model: https://huggingface.co/openaccess-ai-collective/manticore-13b
This quantization was made by using this repository: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
And I used the triton branch with all the gptq implementations available (true_sequential + act_order + groupsize 128)
CUDA_VISIBLE_DEVICES=0 python llama.py ./Manticore-13b-GPTQ-Triton c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors manticore-13b-4bit-128g.safetensors
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- May 21, 2023
- Updated
- May 21, 2023