Downloads · 30 days
5
12% of all-time downloads
Llama-3.2-1B-Instruct-quantized-gptq-4g01
arishiki/Llama-3.2-1B-Instruct-quantized-gptq-4g01
Llama-3.2-1B-Instruct-quantized-gptq-4g01 is a machine learning model from arishiki. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.2.
This model is a quantized version of Llama-3.2-1B-Instruct. Code used for generation is as follows:
All-time downloads
42
Public
Parameters
1.2B
1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1 GB · 98%
Parameter types
How the weights are stored.
I32973M · 79%
Base models
Model card
From the Hugging Face model README
This model is a quantized version of Llama-3.2-1B-Instruct. Code used for generation is as follows:
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig
import torch
model_id = "meta-llama/Llama-3.2-1B-Instruct"
quantization_config = GPTQConfig(
bits=4,
group_size=128,
dataset="c4",
desc_act=False,
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
quant_model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=quantization_config, device_map='auto')
- Type
- llama
- License
- llama3.2
- Created
- Oct 21, 2024
- Updated
- Oct 21, 2024