Downloads · 30 days
35
2% of all-time downloads
Durlabh/gemma-270m-q4-k-m-gguf
gemma-270m-q4-k-m-gguf is a text generation model from Durlabh. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as gemma.
This is a quantized version of Google's Gemma 3 270M instruction-tuned model, optimized for efficient inference on consumer hardware and mobile applications. The model has been converted to GGUF format and quantized u…
Downloads · 30 days
35
2% of all-time downloads
All-time downloads
1.5K
Public
Repo size
253 MB
Likes
1
Public
Click a slice to open those files.
.gguf253 MB · 100%
From the Hugging Face model README
This is a quantized version of Google's Gemma 3 270M instruction-tuned model, optimized for efficient inference on consumer hardware and mobile applications. The model has been converted to GGUF format and quantized using Q4_K_M quantization through llama.cpp, making it perfect for resource-constrained environments.
This model is actively used in a production mobile application available on app stores. The app demonstrates the practical viability of running quantized LLMs on mobile devices while maintaining user privacy through on-device inference. The implementation showcases:
# Download the model
wget https://huggingface.co/Durlabh/gemma-270m-q4-k-m-gguf/resolve/main/gemma-270m-q4-k-m.gguf
# Run inference
./main -m gemma-270m-q4-k-m.gguf -p "Your prompt here"
# Create Modelfile
echo "FROM ./gemma-270m-q4-k-m.gguf" > Modelfile
# Create and run
ollama create gemma-270m-q4 -f Modelfile
ollama run gemma-270m-q4
from llama_cpp import Llama
# Load model
llm = Llama(model_path="gemma-270m-q4-k-m.gguf")
# Generate text
output = llm("Your prompt here", max_tokens=100)
print(output['choices'][0]['text'])
For mobile app development, this model can be integrated using:
| Metric | Original F16 | Q4_K_M | Improvement |
|---|---|---|---|
| Size | ~540MB | 253MB | 53% reduction |
| RAM Usage | ~1GB | ~400MB | 60% reduction |
| Inference Speed | Baseline | ~2x faster | 2x speedup |
| Mobile Performance | Too large | Excellent | ✅ Mobile ready |
Performance tested on various devices including mobile hardware
Important: This model is a derivative of Google's Gemma and is subject to the original licensing terms.
Gemma is provided under and subject to the Gemma Terms of Use.
As per the Gemma Terms of Use, this model cannot be used for:
See the full Prohibited Use Policy for complete details.
This model is used in compliance with:
# Using wget
wget https://huggingface.co/Durlabh/gemma-270m-q4-k-m-gguf/resolve/main/gemma-270m-q4-k-m.gguf
# Using curl
curl -L -o gemma-270m-q4-k-m.gguf https://huggingface.co/Durlabh/gemma-270m-q4-k-m-gguf/resolve/main/gemma-270m-q4-k-m.gguf
# Using huggingface-hub
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(
repo_id="Durlabh/gemma-270m-q4-k-m-gguf",
filename="gemma-270m-q4-k-m.gguf"
)
If you use this model, please cite both the original Gemma work and acknowledge the quantization:
@misc{durlabh-gemma-270m-q4-k-m,
title={Gemma 3 270M Instruction-Tuned Q4_K_M Quantized},
author={Durlabh},
year={2025},
note={Quantized version of Google's Gemma 3 270M instruction-tuned model using llama.cpp Q4_K_M},
url={https://huggingface.co/Durlabh/gemma-270m-q4-k-m-gguf}
}
Original Gemma 3 paper:
@misc{gemma3_2025,
title={Gemma 3: Google's new open model based on Gemini 2.0},
author={Gemma Team},
year={2025},
publisher={Google},
url={https://blog.google/technology/developers/gemma-3/}
}
This is an unofficial quantized version of Gemma 3 created for practical mobile deployment. For official Gemma models, please visit Google's official Gemma page.
The mobile application using this model fully complies with platform guidelines and Gemma licensing requirements.
Ready for production use! This model powers real-world mobile applications while maintaining full compliance with licensing terms.