Downloads · 30 days
309
26% of all-time downloads
gemma-4-12B-it-GGUF
bombman/gemma-4-12B-it-GGUF
gemma-4-12B-it-GGUF is a machine learning model from bombman. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
GGUF quantizations of Google's Gemma 4 12B IT model.
All-time downloads
1.2K
Public
Repo size
101 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf51 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jun 4, 2026
- Updated
- Jun 4, 2026
- SHA
- b785fdd7
Base models
Model card
From the Hugging Face model README
Gemma 4 12B IT GGUF
GGUF quantizations of Google's Gemma 4 12B IT model.
Optimized for local inference with:
- Ollama
- LM Studio
- Jan AI
- llama.cpp
- Open WebUI
- Text Generation WebUI
- KoboldCpp
Downloads
| Quant | Recommended Use |
|---|---|
| Q4_K_S | Smallest size |
| Q4_K_M | ⭐ Recommended |
| Q8_0 | Maximum quality |
| F16 | Further quantization |
Quick Start
Ollama
Create a file named Modelfile
FROM ./gemma-4-12B-it-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER num_ctx 32768
SYSTEM You are a helpful AI assistant.
Create model:
ollama create gemma4-12b -f Modelfile
Run:
ollama run gemma4-12b
LM Studio
- Download a GGUF file.
- Open LM Studio.
- Import the model.
- Load the model.
- Start chatting.
Recommended:
gemma-4-12B-it-Q4_K_M.gguf
Jan AI
- Download a GGUF file.
- Open Jan AI.
- Models → Import Model
- Select the GGUF file.
- Start chatting.
Recommended:
gemma-4-12B-it-Q4_K_M.gguf
Open WebUI
Using Ollama:
ollama create gemma4-12b -f Modelfile
Then select:
gemma4-12b
inside Open WebUI.
llama.cpp
Run:
./build/bin/llama-cli \
-m gemma-4-12B-it-Q4_K_M.gguf \
-cnv
Server:
./build/bin/llama-server \
-m gemma-4-12B-it-Q4_K_M.gguf
OpenAI-compatible endpoint:
http://localhost:8080/v1
Hardware Recommendations
| Quant | RAM |
|---|---|
| Q4_K_S | 10 GB+ |
| Q4_K_M | 12 GB+ |
| Q8_0 | 18 GB+ |
| F16 | 32 GB+ |
Which Version Should I Download?
Most Users
Q4_K_M
Best balance of:
- Quality
- Speed
- Memory usage
Highest Quality
Q8_0
Smallest Size
Q4_K_S
Further Quantization
F16
Credits
- Google for Gemma 4
- ggml-org for llama.cpp
- Ollama
- LM Studio
- Jan AI
- Open WebUI
Disclaimer
This repository contains GGUF conversions only.
All original model weights and intellectual property belong to Google.
- License
- apache-2.0
- Languages
- en
- Created
- Jun 4, 2026
- Updated
- Jun 4, 2026