Downloads · 30 days
388
0% of all-time downloads
williamliao/gemma-4-26B-A4B-it-speculator.eagle3-F16-GGUF
gemma-4-26B-A4B-it-speculator.eagle3-F16-GGUF is a text generation model from williamliao. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
This repository contains GGUF conversions and quantizations of RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3 for use with llama.cpp EAGLE3 speculative decoding.
Downloads · 30 days
388
0% of all-time downloads
All-time downloads
102K
Public
Repo size
3.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf3.4 GB · 100%
From the Hugging Face model README
This repository contains GGUF conversions and quantizations of RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3 for use with llama.cpp EAGLE3 speculative decoding.
[!IMPORTANT] This is not a standalone chat model. It is an EAGLE3 draft/speculator model and must be used together with the matching target/verifier model.
unsloth/gemma-4-26B-A4B-it-GGUFRedHatAI/gemma-4-26B-A4B-it-speculator.eagle3--spec-type draft-eagle3| File | Type | Notes |
|---|---|---|
gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf | EAGLE3 speculator GGUF | Converted from the original RedHatAI safetensors checkpoint |
gemma-4-26B-A4B-it-speculator.eagle3-Q8_0.gguf | Quantized EAGLE3 speculator GGUF | Quantized from the F16 GGUF |
gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf | Quantized EAGLE3 speculator GGUF | Quantized from the F16 GGUF; may be faster for draft decoding |
Example:
llama-server \
-m gemma-4-26B-A4B-it-Q4_K_M.gguf \
-md gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf \
--spec-type draft-eagle3 \
--spec-draft-n-max 4 \
--spec-draft-p-min 0.5 \
-c 32768 \
-ngl 99 \
-fa on
Windows CMD example:
llama-server.exe ^
-m gemma-4-26B-A4B-it-Q4_K_M.gguf ^
-md gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf ^
--spec-type draft-eagle3 ^
--spec-draft-n-max 4 ^
--spec-draft-p-min 0.5 ^
-c 32768 ^
-ngl 99 ^
-fa on
PowerShell example:
.\llama-server.exe `
-m "gemma-4-26B-A4B-it-Q4_K_M.gguf" `
-md "gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf" `
--spec-type draft-eagle3 `
--spec-draft-n-max 4 `
--spec-draft-p-min 0.5 `
-c 32768 `
-ngl 99 `
-fa on
This GGUF file is only the draft/speculator model. You still need a compatible GGUF of the target model, such as unsloth/gemma-4-26B-A4B-it-GGUF.
Do not use this speculator with unrelated models such as Gemma 4 12B, Gemma 4 31B, Gemma 3, Qwen, Llama, Mistral, or other non-matching models. EAGLE3 speculators are target-specific.
Even small differences in the target model, prompt format, quantization, or runtime settings may affect draft acceptance rate and overall speed.
Tested with:
Runtime: llama.cpp with EAGLE3 support
Target model: unsloth/gemma-4-26B-A4B-it-GGUF
Draft model: this EAGLE3 GGUF
Example settings:
--spec-type draft-eagle3--spec-draft-n-max 4--spec-draft-p-min 0.5Local benchmark observations may vary depending on GPU, quantization, context length, batch size, sampling settings, and prompt type.
In local testing, Gemma 4 26B-A4B IT without EAGLE3 already showed strong baseline decoding speed.
With EAGLE3 enabled, the draft acceptance rate was around 0.70 in local testing, with stronger gains on structured or predictable tasks such as:
It was less effective on some open-ended or language-sensitive tasks such as:
On this model, EAGLE3 may be useful for structured output, agent/tool-style responses, code completion, and predictable formats. For general chat, translation, roleplay, or creative writing, the non-speculative baseline may be competitive or more consistent.
On smaller VRAM setups, the extra draft/speculator model may reduce the practical benefit of EAGLE3. In those cases, native MTP models or the base Gemma 4 26B-A4B model without speculative decoding may be more efficient.
Converted with llama.cpp convert_hf_to_gguf.py using the original speculator repository and the matching target model directory.
Example conversion command:
python convert_hf_to_gguf.py \
RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3 \
--outtype f16 \
--target-model-dir gemma-4-26B-A4B-it \
--outfile gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf
PowerShell example:
python .\convert_hf_to_gguf.py `
"E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3" `
--outtype f16 `
--target-model-dir "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it" `
--outfile "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf"
The F16 GGUF can be quantized with llama-quantize.
Q8_0 example:
llama-quantize \
gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf \
gemma-4-26B-A4B-it-speculator.eagle3-Q8_0.gguf \
Q8_0
Q4_K_M example:
llama-quantize \
gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf \
gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf \
Q4_K_M
PowerShell example:
.\llama-quantize.exe `
"E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf" `
"E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf" `
Q4_K_M
Original EAGLE3 speculator model by RedHatAI:
RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3Target GGUF model:
unsloth/gemma-4-26B-A4B-it-GGUFGGUF support and runtime:
ggml-org/llama.cppThis repository is a converted GGUF version of the original speculator model. The original model license and usage terms apply. Please refer to the upstream repositories for full license details.