Downloads · 30 days
12
7% of all-time downloads
Qwen3-32B-GPTQ-8bit-float16
sandman4/Qwen3-32B-GPTQ-8bit-float16
Qwen3-32B-GPTQ-8bit-float16 is a machine learning model from sandman4. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
8-bit quantized with float16 version of Qwen3-32B using gptqmodel.
All-time downloads
166
Public
Parameters
32.8B
35.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors35.1 GB · 100%
Parameter types
How the weights are stored.
I3231.2B · 95%
Base models
Model card
From the Hugging Face model README
Qwen3-32B Quantized Model
8-bit quantized with float16 version of Qwen3-32B using gptqmodel.
Quantization
from datasets import load_dataset
from gptqmodel import GPTQModel, QuantizeConfig
import sys
import torch
model_id = sys.argv[1]
quant_path = "quantized_model"
# Load calibration data (1024 samples from C4)
calibration_dataset = load_dataset(
"allenai/c4",
data_files="en/c4-train.00002-of-01024.json.gz",
split="train"
).select(range(1024))["text"]
# Configure and run quantization
quant_config = QuantizeConfig(bits=8, group_size=128)
model = GPTQModel.load(model_id, quant_config, torch_dtype=torch.float16)
model.quantize(calibration_dataset, batch_size=2)
model.save(quant_path)
License
Apache-v2. See LICENSE.txt
- Type
- qwen3
- License
- apache-2.0
- Created
- May 5, 2025
- Updated
- May 5, 2025