Downloads · 30 days
0
gemma3-12b-SQ-GPTQ
dsuman/gemma3-12b-SQ-GPTQ
gemma3-12b-SQ-GPTQ is a machine learning model from dsuman. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Our submission applies SmoothQuant and GPTQ to the Gemma 3 12B model, producing a W8A8 (8-bit weight, 8-bit activation) compressed model. SmoothQuant is first applied to reduce activation outliers by migrating some of…
Access
Public
Updated Jul 16, 2026
Repo size
13.7 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors13.6 GB · 100%
At a glance
- Access
- Public
- Created
- Jul 16, 2026
- Updated
- Jul 16, 2026
- SHA
- cc004b8b
Model card
From the Hugging Face model README
Gemma3-12B (W8A8): SmoothQuant + GPTQ
Description
Our submission applies SmoothQuant and GPTQ to the Gemma 3 12B model, producing a W8A8 (8-bit weight, 8-bit activation) compressed model. SmoothQuant is first applied to reduce activation outliers by migrating some of the quantization difficulty from activations onto the weights, rescaling so that activations become easier to quantize at the cost of slightly harder-to-quantize weights. The weights are then quantized using GPTQ and activations using RTN quantization.
1. Set up the environment
bash setup.sh
2. Run inference
bash run.sh \
--lang-pair ces-deu \
--batch-size 512 \
--input input.txt \
--output output.txt
Arguments:
| Flag | Description |
|---|---|
--lang-pair | Language pair to translate. This submission only supports ces-deu (Czech → German). |
--batch-size | Number of sequences processed per batch (max_num_seqs in vLLM). |
--input | Path to a plain-text file with one source sentence per line. |
--output | Path to write translations to — one line per input line, in the same order. |
- Created
- Jul 16, 2026
- Updated
- Jul 16, 2026