Downloads · 30 days
874
14% of all-time downloads
RachidAR/gemma-4-12B-it-qat-q4_0-MTP-assistant-gguf
gemma-4-12B-it-qat-q4_0-MTP-assistant-gguf is a machine learning model from RachidAR. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
GGUF quantizations converted from google/gemma-4-12B-it-qat-q40-unquantized-assistant.
Downloads · 30 days
874
14% of all-time downloads
All-time downloads
6.3K
Public
Repo size
1 GB
Likes
3
Public
Click a slice to open those files.
.gguf1 GB · 100%
From the Hugging Face model README
GGUF quantizations converted from google/gemma-4-12B-it-qat-q4_0-unquantized-assistant.
Tested with llama.cpp b9549+ (Gemma 4 MTP support).
Q4_0_Q4emb — recommended for most usersQ8_0 — highest precisiongemma-4-12b-qat-it-assistant-Q4_0-Q6emb.ggufgemma-4-12b-qat-it-assistant-Q4_0_Q4emb.ggufgemma-4-12b-qat-it-assistant-Q8_0.ggufQ4_0_Q4emb keeps token_embd.weight in Q4_0 rather than the higher-precision embedding quantization normally used by llama.cpp.
Initial testing showed similar draft acceptance rates while reducing file size.
llama-server \
-m gemma-4-12B-it-qat-Q4_0.gguf \
-md gemma-4-12b-qat-it-assistant-Q4_0_Q4emb.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2