Downloads · 30 days
6
20% of all-time downloads
llama-3.2-3b-instruct-quantized
hlhsiao/llama-3.2-3b-instruct-quantized
llama-3.2-3b-instruct-quantized is a machine learning model from hlhsiao. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a 3B parameter LLaMA 3.2 Instruct model, quantized with 8-bit weights and activations (w8a8). Model files include safetensors-formatted weights and generationconfig.json.
All-time downloads
30
Public
Parameters
3.6B
6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors6 GB · 100%
Parameter types
How the weights are stored.
I82.8B · 78%
Model card
From the Hugging Face model README
LLaMA 3.2 3B Instruct - Quantized (w8a8)
This is a 3B parameter LLaMA 3.2 Instruct model, quantized with 8-bit weights and activations (w8a8).
Model files include safetensors-formatted weights and generation_config.json.
Files
model-00001-of-00002.safetensors+index.json: quantized model weightsconfig.json+generation_config.json: model and generation configurationrecipe.yaml: quantization configuration (if applicable)
How to load
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("your-username/llama-3.2-3b-instruct-quantized", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("your-username/llama-3.2-3b-instruct-quantized")
- Type
- llama
- Created
- May 29, 2025
- Updated
- May 29, 2025