Downloads · 30 days
10
12% of all-time downloads
Llama-3.2-3B-Instruct-AWQ-4bit
Sumo10/Llama-3.2-3B-Instruct-AWQ-4bit
Llama-3.2-3B-Instruct-AWQ-4bit is a machine learning model from Sumo10. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This is a 4-bit AWQ quantized version of meta-llama/Llama-3.2-3B-Instruct.
All-time downloads
86
Public
Parameters
3.2B
2.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.3 GB · 99%
Parameter types
How the weights are stored.
I322.8B · 88%
Model card
From the Hugging Face model README
Llama 3.2 3B Instruct - AWQ Quantized (4-bit)
This is a 4-bit AWQ quantized version of meta-llama/Llama-3.2-3B-Instruct.
Quantized using:
w_bit: 4q_group_size: 128zero_point: true
Usage
from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer
model = AutoAWQForCausalLM.from_quantized("Sumo10/Llama-3.2-3B-Instruct-AWQ-4bit")
tokenizer = AutoTokenizer.from_pretrained("Sumo10/Llama-3.2-3B-Instruct-AWQ-4bit")
input_ids = tokenizer("What is quantum computing?", return_tensors="pt").input_ids
output = model.generate(input_ids)
print(tokenizer.decode(output[0]))
- Type
- llama
- License
- mit
- Languages
- en
- Created
- Apr 25, 2025
- Updated
- Apr 25, 2025