Downloads · 30 days
79
29% of all-time downloads
manojbillionaire123/Gemma-4-E4B-it-qat-assistant-MTP-Q8_0-GGUF
Gemma-4-E4B-it-qat-assistant-MTP-Q8_0-GGUF is a machine learning model from manojbillionaire123. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as apache-2.0.
This model is a GGUF quantized version of the google/gemma-4-E4B-it-qat-q40-unquantized-assistant base model, converted using llama.cpp's https://github.com/ggml-org/llama.cpp/blob/master/converthftogguf.py script.
Downloads · 30 days
79
29% of all-time downloads
All-time downloads
276
Public
Repo size
98.7 MB
Likes
1
Public
Click a slice to open those files.
.gguf98.7 MB · 100%
From the Hugging Face model README
This model is a GGUF quantized version of the google/gemma-4-E4B-it-qat-q4_0-unquantized-assistant base model, converted using llama.cpp's https://github.com/ggml-org/llama.cpp/blob/master/convert_hf_to_gguf.py script.
google/gemma-4-E4B-it-qat-q4_0-unquantized-assistanthttps://github.com/ggml-org/llama.cpp/blob/master/convert_hf_to_gguf.py| File | Description |
|---|---|
gemma-4-E4B-it-qat-assistant-MTP-Q8_0.gguf | Draft model for speculative decoding (Q8_0) |
Below is the recommended llama-server command to run this model with optimal settings including MTP (Multi-Token Prediction) speculative decoding:
llama-server -hf google/gemma-4-E4B-it-qat-q4_0-gguf \
--temp 1.0 \
--top-p 0.95 \
--top-k 64 \
--ui-mcp-proxy \
-c 64000 \
-fa off \
--jinja \
--metrics \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
--parallel 1 \
--spec-draft-hf amaranus/Gemma-4-E4B-it-qat-assistant-MTP-Q8_0-GGUF
| Parameter | Value | Description |
|---|---|---|
-hf | google/gemma-4-E4B-it-qat-q4_0-gguf | Load model directly from Hugging Face Hub |
--temp | 1.0 | Sampling temperature (higher = more creative) |
--top-p | 0.95 | Nucleus sampling threshold |
--top-k | 64 | Top-K sampling (only consider top 64 tokens) |
--ui-mcp-proxy | - | Enable UI MCP proxy interface |
-c | 64000 | Context length (64K tokens) |
-fa | off | Flash Attention disabled (use standard attention) |
--jinja | - | Enable Jinja2 template processing for chat formats |
--metrics | - | Enable metrics endpoint for monitoring |
--spec-type | draft-mtp | Speculative decoding type: Multi-Token Prediction |
--spec-draft-n-max | 2 | Maximum number of speculative tokens per step |
--parallel | 1 | Parallel sequences (1 = single sequence) |
--model-draft | gemma-4-E4B-it-qat-assistant-MTP-Q8_0.gguf | Path to draft model (higher quality Q8_0) |