Downloads · 30 days
0
gemma3-12b-GPTQ-RTN
dsuman/gemma3-12b-GPTQ-RTN
gemma3-12b-GPTQ-RTN is a machine learning model from dsuman. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
GPTQ-based weight quantization combined with naive round-to-nearest (RTN) activation quantization, producing a W8A8 (8-bit weight, 8-bit activation) compressed model, with inference executed directly in 8-bit precision.
Access
Public
Updated Jul 16, 2026
Repo size
13.7 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors13.6 GB · 100%
At a glance
- Access
- Public
- Created
- Jul 16, 2026
- Updated
- Jul 16, 2026
- SHA
- b1f5b86b
Model card
From the Hugging Face model README
Gemma 3 12B — GPTQ + RTN (W8A8)
Compression methods employed
GPTQ-based weight quantization combined with naive round-to-nearest (RTN) activation quantization, producing a W8A8 (8-bit weight, 8-bit activation) compressed model, with inference executed directly in 8-bit precision.
1. Set up the environment
bash setup.sh
2. Run inference
bash run.sh \
--lang-pair ces-deu \
--batch-size 512 \
--input input.txt \
--output output.txt
Arguments:
| Flag | Description |
|---|---|
--lang-pair | Language pair to translate. This submission only supports ces-deu (Czech → German). |
--batch-size | Number of sequences processed per batch (max_num_seqs in vLLM). |
--input | Path to a plain-text file with one source sentence per line. |
--output | Path to write translations to — one line per input line, in the same order. |
- Created
- Jul 16, 2026
- Updated
- Jul 16, 2026