gemma-4-e4b-it-qat

Model Card for gemma-4-E4B-it (QAT, GGUF)

A GGUF repack of Google's quantization-aware-trained gemma-4-E4B-it build, packaged to run on device. It carries the text, vision and audio towers in one self-contained file: the tokenizer and chat template are embedded, so nothing ships alongside it and nothing calls out to a server at inference time.

This repository holds the repacked weights only, not training data or the original checkpoint.

File format

.ggxf is GGUF plus the tensors and metadata keys this project adds, so these files load in Gadeon and are not expected to load in other GGUF readers.

Model Details

Model Description

gemma-4-E4B is a multimodal decoder with a per-layer-embedding (PLE) design: alongside the usual token embedding, each layer receives its own 256-wide input gathered from a large per-layer table, gated and projected into the hidden stream. Attention alternates five sliding-window layers to one full-attention layer, the two using different head widths and RoPE bases, and the upper eighteen layers share their neighbours' key/value state instead of computing their own. A vision encoder and a Conformer audio encoder project into the same embedding space.

The upstream checkpoint is quantization-aware TRAINED, not post-hoc quantized: Google trained it while simulating low-bit error, which is why its two-bit layers hold up where a post-hoc conversion of the same layers does not. E4B spends its bits differently from E2B: every feed-forward block stays at four bits, while the two embedding tables and the output projection drop to two.

• Developed by: Google (base model); GGUF repack by leok7v
• Model type: Multimodal (text, image, audio) causal language model
• Language(s): the languages of the base model
• License: Apache 2.0, inherited from the base model; see the Gemma 4 license
• Repacked from model: google/gemma-4-E4B-it-qat-mobile-transformers

Model Sources

• Repository (this build): https://huggingface.co/leok7v/gemma-4-e4b-it-qat
• Base model: google/gemma-4-E4B-it-qat-mobile-transformers

Uses

Direct Use

On-device assistants and chat in offline or privacy-sensitive settings where prompts must not leave the device.

Out-of-Scope Use

Not for tasks requiring verified factual accuracy, and not a basis for legal, medical, financial, or safety decisions. Not intended for high-stakes automated decisions without human review.

Bias, Risks, and Limitations

A language model predicts plausible text; it does not verify facts and has no understanding of truth. Output can be confident and still be wrong, incomplete, or biased. The base model's biases and limitations carry over; see the base model card.

Recommendations

Treat every response critically and check important output against a trusted source.

How to Get Started with the Model

This file does not load in upstream llama.cpp. 111 of its tensors use Q2_0, a block type that exists in the PrismML/Gadeon ggml lineage and not in upstream ggml, so a tool that does not know Q2_0 will fail to read it. It is published for runtimes that carry that type.

The tokenizer (262144 tokens, 514906 merges, byte fallback and metaspace) and the chat template are embedded in the file, so a runtime reads both from the GGUF rather than from sibling files.

Technical Specifications

Model Architecture and Objective

Weight format

3.795 GB, 1149 tensors, GGUF v3, 16384-byte tensor alignment.

The integer codes transfer from the QAT checkpoint exactly rather than being re-quantized: its INT4 codebook and offset are Q4_0's, and its INT2 codes are Q2_0's under a negation. The only numerical difference from the upstream weights is a per-block fp32 to fp16 scale cast.

Citation

Please cite the upstream Gemma model. See the base model card at google/gemma-4-E4B-it-qat-mobile-transformers for the canonical reference and license.

Model Card Contact

leok7v on Hugging Face.