Google-Gemma-4-31B-It-GGUF

This repository contains quantized GGUF formats of the Google Gemma 4 31B Instruction-Tuned model, published and heavily optimized by CQ Systems.

⚡ Mac Neo & Apple Silicon Compatibility

These files are optimized for local inference on professional hardware. Due to the 31-billion parameter size of this model, it is recommended to run these on machines with substantial unified memory (such as high-end MacBook Pros, Mac Studios, or Mac Pros).

By utilizing Apple Silicon's advanced unified memory matrix and native processing capabilities via Metal, these quantizations deliver fast, on-device reasoning and frontier-level cognition without relying on cloud infrastructure.

📦 Available Files

We provide multiple versions of the model, allowing you to scale based on your available unified memory (RAM/VRAM) constraints and latency requirements.

🚀 How to Use

Using llama.cpp

You can run this model directly in your terminal using the compiled llama.cpp CLI tool. Ensure you have built llama.cpp with Metal support for Macs.

# Basic chat launch with the recommended Q4_K model
./llama-cli -m CQ-Google-Gemma-4-31B-f16-Q4_K.gguf \
-c 8192 \
-n 1024 \
-p "<|think|>\nAnalyze the system architecture.\n<|channel>thought\n"

Using LM Studio / Ollama

1. Download the .gguf file matching your system's capabilities.
2. LM Studio: Drag and drop the downloaded file into your LM Studio model folder, or use the local import feature.
3. Ollama: Create a Modelfile with the line FROM ./CQ-Google-Gemma-4-31B-f16-Q4_K.gguf, then run ollama create cq-gemma-31b -f Modelfile.

(Note: While the base Gemma 4 31B supports audio and image inputs natively, standard GGUF text-generation primarily supports text out-of-the-box. Multimodal capabilities in GGUF require an accompanying mmproj encoder file).

🧠 About the Base Model: Gemma 4 31B

Gemma 4 is a family of open models built by Google DeepMind. The 31B variant is the flagship heavyweight model in the family, designed to handle highly complex reasoning, advanced coding, and nuanced logic comparable to frontier closed-source models.

• Total Parameters: 31B Dense
• Context Length: 128K tokens
• Supported Modalities (Base Model): Text, Image, Audio
• System Prompts: Gemma 4 uses standard system, assistant, and user roles natively.

Thinking Mode Configuration

Gemma 4 models are designed as highly capable reasoners. To enable the built-in reasoning capabilities, include the <|think|> token at the start of your system prompt.

When thinking is enabled, the model will output its internal reasoning followed by the final answer using this structural formatting:

<|channel>thought\n**[Internal reasoning]<channel|>[Final answer]**

📜 License & Acknowledgements

• Base Model: Google DeepMind
• License: Gemma License
• Quantization & Optimization: CQ Systems Team

For full benchmark details, evaluations, ethical considerations, and safety metrics, please refer to the original Google Gemma 4 Documentation.