Downloads · 30 days
324
16% of all-time downloads
g0chu/gemma-4-31B-it-qat-q4_0-unquantized-assistant-q8_0-gguf
gemma-4-31B-it-qat-q4_0-unquantized-assistant-q8_0-gguf is a machine learning model from g0chu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Use as draft model for gemma-4-31B-it-qat in llama.cpp.
Downloads · 30 days
324
16% of all-time downloads
All-time downloads
2.1K
Public
Repo size
980 MB
Likes
2
Public
Click a slice to open those files.
.gguf515 MB · 100%
From the Hugging Face model README
Use as draft model for gemma-4-31B-it-qat in llama.cpp.
Tested with:
/home/gochu/workspace/repos/llama.cpp-gemma4-mtp/build/bin/llama-server \
-m /home/gochu/workspace/models/gguf/gemma/qat/gemma-4-31B_q4_0-it.gguf \
-md /home/gochu/workspace/models/gguf/gemma/draft/gemma-4-31B-it-qat-q4_0-unquantized-assistant-q8_0.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 3 \
--host 0.0.0.0 \
--port 8080 \
-c 128000 \
-ngl 99 \
-sm tensor \
-b 2048 \
-ub 512 \
-fa on \
-np 1 \
--jinja \
--chat-template-kwargs '{"enable_thinking":true}'