gemma-4-E4B-it-Q6_K

gemma-4-E4B-it — GGUF (Q6_K)

---

📊 Performance Metrics

• Hardware: AMD EPYC 7B12 (4 vCPUs)
• Size: 5.79 GB
• Speed (Generation): 3.86 tokens/sec
• Speed (Prompt): 10.30 tokens/sec
• KV Cache Usage: 0.0143 GB
• Quantization: Q6_K

---

🔷 Model Overview

This repository contains a GGUF quantized version of:

• Base Model: gemma-4-E4B-it
• Format: GGUF (optimized for llama.cpp inference)
• Precision: Q6_K
• Efficiency Score: 0.6666 (TPS/GB)

GGUF format provides:

• Fast loading via memory mapping
• Single-file model distribution
• Cross-platform compatibility
• Efficient inference with llama.cpp

---

📦 Files

---

⚙️ Technical Details

---

⚡ Why GGUF?

GGUF is designed for efficient inference:

• Optimized for llama.cpp
• Supports CPU and GPU inference
• Single-file deployment
• Memory-mapped loading for speed
• Ideal for edge / local environments

---

⚠️ License & Usage

This is a converted derivative model.

• You must comply with the original model license of gemma-4-E4B-it
• This is not an official release
• No additional rights are granted
• Original ownership remains with the base model creator

---

🚀 Quick Start (llama.cpp)

./llama-cli -m gemma-4-E4B-it-Q6_K.gguf -p "Explain AI simply"