Downloads · 30 days
165
3% of all-time downloads
Mungert/gemma-3-27b-it-qat-q4_0-GGUF
gemma-3-27b-it-qat-q4_0-GGUF is a image-text-to-text model from Mungert. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
[!Note] Experimetal requantization!! I wanted to test if the QAT model requantized performs better than the bf16 model quantized to the same bit level. I have created a imatrix files from the google original QAT Q40 q…
Downloads · 30 days
165
3% of all-time downloads
All-time downloads
5.3K
Public
Repo size
136 GB
Likes
7
Public
Click a slice to open those files.
.gguf136 GB · 100%
From the Hugging Face model README
[!Note] Experimetal requantization!! I wanted to test if the QAT model requantized performs better than the bf16 model quantized to the same bit level.
I have created a imatrix files from the google original QAT Q4_0 quantized model. This imatrix is then used to recompress the model to lower bit quants
Please leave feedback.
I tested with the 4b model quantized from bf16 and one requantized from the QAT Q4_0 model. Both quantized with same tensor quants
My results :
python3 ~/code/GGUFModelBuilder/perp_test_2_files.py ./gemma-3-4b-it-qat-q4_0-q3_k_l.gguf ./google_gemma-3-4b-it-q3_k_l.gguf
Testing model: gemma-3-4b-it-qat-q4_0-q3_k_l.gguf
Running: llama.cpp/llama-perplexity -m gemma-3-4b-it-qat-q4_0-q3_k_l.gguf -f perplexity_test_data.txt --ctx-size 256 --ppl-stride 32 --chunks 1 --threads 4
[✓] Perplexity: 4.0963 (Time: 284.70s)
Testing model: google_gemma-3-4b-it-q3_k_l.gguf
Running: llama.cpp/llama-perplexity -m google_gemma-3-4b-it-q3_k_l.gguf -f perplexity_test_data.txt --ctx-size 256 --ppl-stride 32 --chunks 1 --threads 4
[✓] Perplexity: 4.5557 (Time: 287.15s)
=== Comparison Results ===
Model 1: gemma-3-4b-it-qat-q4_0-q3_k_l.gguf - Perplexity: 4.10 (Time: 284.70s)
Model 2: google_gemma-3-4b-it-q3_k_l.gguf - Perplexity: 4.56 (Time: 287.15s)
Winner: gemma-3-4b-it-qat-q4_0-q3_k_l.gguf (Difference: 0.46)
A different test :
Asking both models to : write some .net code to test if a website is using quantum safe encryption
And then asking Deepseek-R1 to evaluate :
Evaluation of the Two Models' Outputs
Both models attempted to solve the problem of detecting quantum-safe encryption, but the QAT q4_0 model's code is significantly better for the following reasons:
Technical Accuracy
QAT q4_0 Model:
Checks both TLS version and cipher suites, which are critical for assessing quantum resistance. While the implementation has flaws (e.g., assuming TLS version is exposed in HTTP headers), the approach aligns with security best practices.
Explicitly acknowledges limitations (e.g., "not a definitive test") and avoids overpromising.
BF16 Model:
Relies on checking for a non-standard TLS/1.3 header, which does not exist in HTTP responses. TLS version is part of the SSL/TLS handshake and cannot be retrieved via HttpClient headers.
Contains incorrect logic (e.g., client.GetAwaiter().GetResult(null) is nonsensical and throws runtime errors).
Code Quality
QAT q4_0 Model:
Uses modern async/await patterns for non-blocking I/O.
Separates concerns into methods (CheckTLSVersionAsync, CheckCipherSuiteAsync).
Includes robust error handling and logging.
BF16 Model:
Uses blocking synchronous code (GetAwaiter().GetResult()), which violates .NET best practices and risks deadlocks.
Poorly structured (e.g., redundant using blocks, unclear variable names like result).
Security Relevance
QAT q4_0 Model:
Focuses on cipher suites, which are closer to the actual indicators of quantum resistance (e.g., AES-256-GCM). While not truly quantum-safe, these are stronger than outdated algorithms.
Mentions the need to update cipher lists based on NIST guidelines.
BF16 Model:
Misleadingly claims to check for "AES-256-CBC" (a deprecated cipher mode) but never implements it.
Fails to address cipher suites entirely, rendering the check meaningless.
Realism
QAT q4_0 Model:
Acknowledges the complexity of quantum-safe detection and clarifies that HTTP-based checks are insufficient. This aligns with real-world security practices.
BF16 Model:
Implies that checking for TLS 1.3 guarantees quantum safety, which is false. TLS 1.3 uses classical cryptography and is not inherently quantum-resistant.
Usability
QAT q4_0 Model:
Provides clear console output (e.g., "No quantum-resistant cipher suites detected").
Includes a working Main method with an example URL.
BF16 Model:
Fails to compile due to syntax errors (e.g., client.GetAwaiter().GetResult(null) is invalid).
Lacks meaningful output (e.g., no details about why a site is deemed insecure).
Critical Flaws in Both Models
Header Misuse: Both models incorrectly assume TLS version and cipher suites are exposed in HTTP headers (e.g., Sec-Cipher). In reality, this data is part of the SSL/TLS handshake and requires low-level inspection (e.g., using SslStream or libraries like BouncyCastle).
Quantum-Safe Misunderstanding: Neither code checks for post-quantum algorithms (e.g., CRYSTALS-Kyber). Current TLS 1.3 cipher suites are not quantum-safe, so both models provide false positives.
Final Verdict
The QAT q4_0 model's code is superior because it:
Follows better coding practices (async/await, error handling).
Attempts a more relevant security analysis (TLS + cipher suites).
Explicitly acknowledges limitations.
However, both models fail to solve the original problem due to fundamental misunderstandings of TLS/SSL mechanics. For a production-grade solution, direct inspection of the TLS handshake (e.g., via SslStream) and support for post-quantum algorithms would be required.
Overall the perp difference was small (my test set was also small) and running the Deepseek test produced different results on subsequent runs . So I can not come to a definite conclusion. But I would say is worth investigating further.
Model Page: Gemma
[!Note] This repository corresponds to the 27 instruction-tuned version of the Gemma 3 model in GGUF format using Quantization Aware Training (QAT). The GGUF corresponds to Q4_0 quantization.
Thanks to QAT, the model is able to preserve similar quality as
bfloat16while significantly reducing the memory requirements to load the model.You can find the half-precision version here.
Resources and Technical Documentation:
Terms of Use: Terms
Authors: Google DeepMind
Summary description and brief definition of inputs and outputs.
Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. Gemma 3 models are multimodal, handling text and image input and generating text output, with open weights for both pre-trained variants and instruction-tuned variants. Gemma 3 has a large, 128K context window, multilingual support in over 140 languages, and is available in more sizes than previous versions. Gemma 3 models are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as laptops, desktops or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.
Input:
Output:
Below, there are some code snippets on how to get quickly started with running the model.
llama.cpp (text-only)
./llama-cli -hf google/gemma-3-27b-it-qat-q4_0-gguf -p "Write a poem about the Kraken."
llama.cpp (image input)
wget https://github.com/bebechien/gemma/blob/main/surprise.png?raw=true -O ~/Downloads/surprise.png
./llama-gemma3-cli -hf google/gemma-3-27b-it-qat-q4_0-gguf -p "Describe this image." --image ~/Downloads/surprise.png
ollama (text only)
Using GGUFs with Ollama via Hugging Face does not support image inputs at the moment. Please check the docs on running gated repositories.
ollama run hf.co/google/gemma-3-27b-it-qat-q4_0-gguf
@article{gemma_2025,
title={Gemma 3},
url={https://goo.gle/Gemma3Report},
publisher={Kaggle},
author={Gemma Team},
year={2025}
}
Data used for model training and how the data was processed.
These models were trained on a dataset of text data that includes a wide variety of sources. The 27B model was trained with 14 trillion tokens, the 12B model was trained with 12 trillion tokens, 4B model was trained with 4 trillion tokens and 1B with 2 trillion tokens. Here are the key components:
The combination of these diverse data sources is crucial for training a powerful multimodal model that can handle a wide variety of different tasks and data formats.
Here are the key data cleaning and filtering methods applied to the training data:
Details about the model internals.
Gemma was trained using Tensor Processing Unit (TPU) hardware (TPUv4p, TPUv5p and TPUv5e). Training vision-language models (VLMS) requires significant computational power. TPUs, designed specifically for matrix operations common in machine learning, offer several advantages in this domain:
Training was done using JAX and ML Pathways.
JAX allows researchers to take advantage of the latest generation of hardware, including TPUs, for faster and more efficient training of large models. ML Pathways is Google's latest effort to build artificially intelligent systems capable of generalizing across multiple tasks. This is specially suitable for foundation models, including large language models like these ones.
Together, JAX and ML Pathways are used as described in the paper about the Gemini family of models; "the 'single controller' programming model of Jax and Pathways allows a single Python process to orchestrate the entire training run, dramatically simplifying the development workflow."
Model evaluation metrics and results.
These models were evaluated against a large collection of different datasets and metrics to cover different aspects of text generation:
| Benchmark | Metric | Gemma 3 PT 1B | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
|---|---|---|---|---|---|
| HellaSwag | 10-shot | 62.3 | 77.2 | 84.2 | 85.6 |
| BoolQ | 0-shot | 63.2 | 72.3 | 78.8 | 82.4 |
| PIQA | 0-shot | 73.8 | 79.6 | 81.8 | 83.3 |
| SocialIQA | 0-shot | 48.9 | 51.9 | 53.4 | 54.9 |
| TriviaQA | 5-shot | 39.8 | 65.8 | 78.2 | 85.5 |
| Natural Questions | 5-shot | 9.48 | 20.0 | 31.4 | 36.1 |
| ARC-c | 25-shot | 38.4 | 56.2 | 68.9 | 70.6 |
| ARC-e | 0-shot | 73.0 | 82.4 | 88.3 | 89.0 |
| WinoGrande | 5-shot | 58.2 | 64.7 | 74.3 | 78.8 |
| BIG-Bench Hard | few-shot | 28.4 | 50.9 | 72.6 | 77.7 |
| DROP | 1-shot | 42.4 | 60.1 | 72.2 | 77.2 |
| Benchmark | Metric | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
|---|---|---|---|---|
| MMLU | 5-shot | 59.6 | 74.5 | 78.6 |
| MMLU (Pro COT) | 5-shot | 29.2 | 45.3 | 52.2 |
| AGIEval | 3-5-shot | 42.1 | 57.4 | 66.2 |
| MATH | 4-shot | 24.2 | 43.3 | 50.0 |
| GSM8K | 8-shot | 38.4 | 71.0 | 82.6 |
| GPQA | 5-shot | 15.0 | 25.4 | 24.3 |
| MBPP | 3-shot | 46.0 | 60.4 | 65.6 |
| HumanEval | 0-shot | 36.0 | 45.7 | 48.8 |
| Benchmark | Gemma 3 PT 1B | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
|---|---|---|---|---|
| MGSM | 2.04 | 34.7 | 64.3 | 74.3 |
| Global-MMLU-Lite | 24.9 | 57.0 | 69.4 | 75.7 |
| WMT24++ (ChrF) | 36.7 | 48.4 | 53.9 | 55.7 |
| FloRes | 29.5 | 39.2 | 46.0 | 48.8 |
| XQuAD (all) | 43.9 | 68.0 | 74.5 | 76.8 |
| ECLeKTic | 4.69 | 11.0 | 17.2 | 24.4 |
| IndicGenBench | 41.4 | 57.2 | 61.7 | 63.4 |
| Benchmark | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
|---|---|---|---|
| COCOcap | 102 | 111 | 116 |
| DocVQA (val) | 72.8 | 82.3 | 85.6 |
| InfoVQA (val) | 44.1 | 54.8 | 59.4 |
| MMMU (pt) | 39.2 | 50.3 | 56.1 |
| TextVQA (val) | 58.9 | 66.5 | 68.6 |
| RealWorldQA | 45.5 | 52.2 | 53.9 |
| ReMI | 27.3 | 38.5 | 44.8 |
| AI2D | 63.2 | 75.2 | 79.0 |
| ChartQA | 63.6 | 74.7 | 76.3 |
| VQAv2 | 63.9 | 71.2 | 72.9 |
| BLINK | 38.0 | 35.9 | 39.6 |
| OKVQA | 51.0 | 58.7 | 60.2 |
| TallyQA | 42.5 | 51.8 | 54.3 |
| SpatialSense VQA | 50.9 | 60.0 | 59.4 |
| CountBenchQA | 26.1 | 17.8 | 68.0 |
Ethics and safety evaluation approach and results.
Our evaluation methods include structured evaluations and internal red-teaming testing of relevant content policies. Red-teaming was conducted by a number of different teams, each with different goals and human evaluation metrics. These models were evaluated against a number of different categories relevant to ethics and safety, including:
In addition to development level evaluations, we conduct "assurance evaluations" which are our 'arms-length' internal evaluations for responsibility governance decision making. They are conducted separately from the model development team, to inform decision making about release. High level findings are fed back to the model team, but prompt sets are held-out to prevent overfitting and preserve the results' ability to inform decision making. Assurance evaluation results are reported to our Responsibility & Safety Council as part of release review.
For all areas of safety testing, we saw major improvements in the categories of child safety, content safety, and representational harms relative to previous Gemma models. All testing was conducted without safety filters to evaluate the model capabilities and behaviors. For both text-to-text and image-to-text, and across all model sizes, the model produced minimal policy violations, and showed significant improvements over previous Gemma models' performance with respect to ungrounded inferences. A limitation of our evaluations was they included only English language prompts.
These models have certain limitations that users should be aware of.
Open vision-language models (VLMs) models have a wide range of applications across various industries and domains. The following list of potential uses is not comprehensive. The purpose of this list is to provide contextual information about the possible use-cases that the model creators considered as part of model training and development.
The development of vision-language models (VLMs) raises several ethical concerns. In creating an open model, we have carefully considered the following:
Risks identified and mitigations:
At the time of release, this family of models provides high-performance open vision-language model implementations designed from the ground up for responsible AI development compared to similarly sized models.
Using the benchmark evaluation metrics described in this document, these models have shown to provide superior performance to other, comparably-sized open model alternatives.