Downloads · 30 days
147
2% of all-time downloads
Radamanthys11/Gemma-4-26B-A4B-it-assistant-GGUF
Gemma-4-26B-A4B-it-assistant-GGUF is a machine learning model from Radamanthys11. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as gemma.
This repository contains GGUF quantizations of google/gemma-4-26B-A4B-it-assistant to use as MTP.
Downloads · 30 days
147
2% of all-time downloads
All-time downloads
8.2K
Public
Repo size
1.3 GB
Likes
4
Public
Click a slice to open those files.
.gguf1.3 GB · 100%
From the Hugging Face model README
This repository contains GGUF quantizations of google/gemma-4-26B-A4B-it-assistant to use as MTP.
Do not use with llama.cpp as they do not offer support. This model is only compatible with ik_llama.cpp.
Use only with PR 1744, which enables the gemma 4 MTP to work, otherwise you will encounter errors such as the model failing to load.
New: PR is merged.
The assistant is a draft model that requires the main Gemma 4 26B-A4B target GGUF alongside it.
./build/bin/llama-server \
--model google_gemma-4-26B-A4B-it-Q8_0.gguf \
--ctx-size 32768 -ctk q8_0 -ctv q8_0 --n-gpu-layers 99 \
-b 1024 -ub 1024 --jinja \
--spec-type mtp -md gemma-4-26B-A4B-it-assistant-Q8_0.gguf -ngld 99 \
--draft-max 3 --draft-p-min 0.0
Note:
--draft-max 3is a good starting point.