Downloads · 30 days
166
4% of all-time downloads
RachidAR/gemma-4-31B-it-qat-Q4_0-Q4emb-MTP-assistant-gguf
gemma-4-31B-it-qat-Q4_0-Q4emb-MTP-assistant-gguf is a machine learning model from RachidAR. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
GGUF quantizations converted from google/gemma-4-31B-it-qat-q40-unquantized-assistant.
Downloads · 30 days
166
4% of all-time downloads
All-time downloads
4.7K
Public
Repo size
1.6 GB
Likes
5
Public
Click a slice to open those files.
.gguf1.6 GB · 100%
From the Hugging Face model README
GGUF quantizations converted from google/gemma-4-31B-it-qat-q4_0-unquantized-assistant.
Tested with llama.cpp b9549+ (Gemma 4 MTP support).
Q4_0-Q4emb — recommended for most usersQ4_0-Q6emb — standard llama.cpp quantizationgemma-4-31b-qat-it-assistant-bf16.ggufgemma-4-31b-qat-it-assistant-Q4_0-Q4emb.ggufgemma-4-31b-qat-it-assistant-Q4_0-Q6emb.ggufQ4_0-Q4emb keeps token_embd.weight in Q4_0, making it closer to the original QAT layout.
This follows a similar approach used by recent Unsloth Gemma QAT releases. Initial testing showed comparable draft acceptance rates while reducing model size.
llama-server \
-m gemma-4-31B-it-qat-Q4_0.gguf \
-md gemma-4-31b-qat-it-assistant-Q4_0-Q4emb.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2
Recommended values:
--spec-draft-n-max 2 for general use--spec-draft-n-max 3 for coding workloads--spec-draft-n-max 4 may help on highly repetitive tasks