Downloads · 30 days
163
5% of all-time downloads
Janvitos/gemma-4-31B-it-qat-assistant-MTP-Q8_0-GGUF
gemma-4-31B-it-qat-assistant-MTP-Q8_0-GGUF is a machine learning model from Janvitos. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
This repository contains a GGUF conversion of the official Google Gemma 4 31B IT QAT assistant/drafter checkpoint.
Downloads · 30 days
163
5% of all-time downloads
All-time downloads
3.2K
Public
Repo size
980 MB
Likes
5
Public
Click a slice to open those files.
.gguf515 MB · 100%
From the Hugging Face model README
This repository contains a GGUF conversion of the official Google Gemma 4 31B IT QAT assistant/drafter checkpoint.
google/gemma-4-31B-it-qat-q4_0-unquantized-assistantgemma-4-31B-it-qat-assistant-MTP-Q8_0.ggufQ8_0This is not a standalone chat model. It is an assistant / drafter / MTP head intended to be used together with a matching Gemma 4 31B IT QAT target model for speculative decoding.
| File | Description |
|---|---|
gemma-4-31B-it-qat-assistant-MTP-Q8_0.gguf | Q8_0 GGUF conversion of the Gemma 4 31B QAT assistant checkpoint |
Converted from the official Google checkpoint:
google/gemma-4-31B-it-qat-q4_0-unquantized-assistant
This GGUF is a draft / assistant / MTP model, not a standalone chat model. It must be loaded together with a matching Gemma 4 31B IT QAT target model.
llama-server example:
llama-server \
-m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf \
--model-draft gemma-4-31B-it-qat-assistant-MTP-Q8_0.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 4
Converted with llama.cpp using Gemma 4 assistant / MTP support:
python convert_hf_to_gguf.py \
gemma-4-31B-it-qat-q4_0-unquantized-assistant \
--outfile gemma-4-31B-it-qat-assistant-MTP-Q8_0.gguf \
--outtype q8_0
In local testing with a Gemma 4 31B IT QAT target GGUF and upstream llama.cpp Gemma 4 MTP support, --spec-draft-n-max 4 gave a good balance of throughput and draft acceptance.
This file was created to make the official QAT assistant/drafter checkpoint usable with llama.cpp's Gemma 4 MTP / speculative decoding path.
This model is intended for users who already have a compatible Gemma 4 31B IT QAT target GGUF and want to enable speculative decoding with the matching QAT assistant head.
This model is a converted derivative of Google's official Gemma 4 QAT assistant checkpoint.
Gemma 4 is released under the Apache 2.0 license. Users should also review the original Google model page and comply with any applicable terms associated with the source checkpoint:
google/gemma-4-31B-it-qat-q4_0-unquantized-assistant