Downloads · 30 days
1K
45% of all-time downloads
Llama-3-8B-Instruct-abliterated-v2-bnb-4bit
huggingkot/Llama-3-8B-Instruct-abliterated-v2-bnb-4bit
Llama-3-8B-Instruct-abliterated-v2-bnb-4bit is a machine learning model from huggingkot. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a converted weight from Llama-3-8B-Instruct-abliterated-v2 model in unsloth 4-bit dynamic quant using this collab notebook.
All-time downloads
2.3K
Public
Parameters
8.2B
5.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.7 GB · 100%
Parameter types
How the weights are stored.
U87.2B · 87%
Base models
Model card
From the Hugging Face model README
This is a converted weight from Llama-3-8B-Instruct-abliterated-v2 model in unsloth 4-bit dynamic quant using this collab notebook.
About this Conversion
This conversion uses Unsloth to load the model in 4-bit format and force-save it in the same 4-bit format.
How 4-bit Quantization Works
- The actual 4-bit quantization is handled by BitsAndBytes (bnb), which works under Torch via AutoGPTQ or BitsAndBytes.
- Unsloth acts as a wrapper, simplifying and optimizing the process for better efficiency.
This allows for reduced memory usage and faster inference while keeping the model compact.
- Created
- Mar 6, 2025
- Updated
- Mar 6, 2025