Downloads · 30 days
2
8% of all-time downloads
Llama-3.2-1B-Instruct-quantized-gptq-8g01
arishiki/Llama-3.2-1B-Instruct-quantized-gptq-8g01
Llama-3.2-1B-Instruct-quantized-gptq-8g01 is a machine learning model from arishiki. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.2.
This model is a quantized version of Llama-3.2-1B-Instruct. The quantization was done using AutoGPTQ (https://github.com/AutoGPTQ/AutoGPTQ). Code used for generation is as follows:
All-time downloads
24
Public
Parameters
1.2B
1.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.5 GB · 99%
Parameter types
How the weights are stored.
I32973M · 79%
Base models
Model card
From the Hugging Face model README
This model is a quantized version of Llama-3.2-1B-Instruct. The quantization was done using AutoGPTQ (https://github.com/AutoGPTQ/AutoGPTQ). Code used for generation is as follows:
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig
import torch
model_id = "meta-llama/Llama-3.2-1B-Instruct"
quantization_config = GPTQConfig(
bits=8,
group_size=128,
dataset="c4",
desc_act=False,
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
quant_model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=quantization_config, device_map='auto')
- Type
- llama
- License
- llama3.2
- Created
- Dec 13, 2024
- Updated
- Dec 13, 2024