Downloads · 30 days
21
3% of all-time downloads
LlamaGuard-7b-GPTQ-4bit-128g-actorder_True
SebastianSchramm/LlamaGuard-7b-GPTQ-4bit-128g-actorder_True
LlamaGuard-7b-GPTQ-4bit-128g-actorder_True is a text generation model from SebastianSchramm. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
The model meta-llama/LlamaGuard-7b was quantized to 4bit, groupsize 128, and act-order=True with auto-gptq integration in transformers (https://huggingface.co/blog/gptq-integration).
All-time downloads
768
Public
Parameters
6.7B
3.9 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors3.9 GB · 100%
Parameter types
How the weights are stored.
I326.5B · 96%
Try a prompt
Base models
Model card
From the Hugging Face model README
Quantized version of meta-llama/LlamaGuard-7b
Model Description
The model meta-llama/LlamaGuard-7b was quantized to 4bit, group_size 128, and act-order=True with auto-gptq integration in transformers (https://huggingface.co/blog/gptq-integration).
Evaluation
To evaluate the qunatized model and compare it with the full precision model, I performed binary classification on the "toxicity" label from the ~5k samples test set of lmsys/toxic-chat.
📊 Full Precision Model:
Average Precision Score: 0.3625
📊 4-bit Quantized Model:
Average Precision Score: 0.3450
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Languages
- en
- Created
- Dec 8, 2023
- Updated
- Dec 19, 2023