Downloads · 30 days
8
7% of all-time downloads
gemma-3-12b-it
jordigonzm/gemma-3-12b-it
gemma-3-12b-it is a machine learning model from jordigonzm. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
114
Public
Repo size
12.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf12.5 GB · 100%
At a glance
- Access
- Public
- Created
- Nov 11, 2025
- Updated
- Nov 11, 2025
- SHA
- e40e9277
Model card
From the Hugging Face model README
Run-adjusted to 16GB VRAM
- KV cache quantitative and context 8192
./main -m ./gemma-3-12b-it-q8_0.gguf \
-c 8192 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
-ngl 999 \
--batch 96
Key notes:
- ngl = nº of layers in VRAM. Upload it as far as it does not give OOM. In 12B it is usually over 28–34 with Q8_0 in 16 GB if the context is large.
If OOM with 8192, go down in this order:
- ngl (e.g., 24).
- batch (e.g., 64).
- c (e.g. 6144 or 4096).
The quantified KV (--cache-type-k/--cache-type-v q8_0) reduces a lot of VRAM for long contexts, at the cost of some performance. It's normal.
- Created
- Nov 11, 2025
- Updated
- Nov 11, 2025