Downloads · 30 days
34
4% of all-time downloads
unofficial-jona/MedGemma-27B-IT-GGUF
MedGemma-27B-IT-GGUF is a image-text-to-text model from unofficial-jona. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as other.
This repository contains a 4-bit GGUF quantization of Google's MedGemma 27B IT, created for local inference with llama.cpp and compatible applications such as LM Studio.
Downloads · 30 days
34
4% of all-time downloads
All-time downloads
805
Public
Repo size
17.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf17.4 GB · 100%
From the Hugging Face model README
This repository contains a 4-bit GGUF quantization of Google's MedGemma 27B IT, created for local inference with llama.cpp and compatible applications such as LM Studio.
It is a multimodal model. Download both files to use image inputs:
| File | Purpose | Approximate disk size |
|---|---|---|
MedGemma-27B-IT-Q4_K_M.gguf | Main language model, 4-bit Q4_K_M quantization | ~16 GB |
mmproj-MedGemma-27B-IT-F16.gguf | Medical vision encoder/projector (kept at F16) | ~1 GB |
Q4_K_M is a quality-oriented 4-bit quantization. It substantially reduces memory use relative to the original BF16 checkpoint while preserving the model's instruction-following capability. This is a lossy conversion; validate outputs for your intended task.
Use a recent build of llama.cpp with Gemma 3 multimodal support.
| Workload | Practical hardware guidance |
|---|---|
| Text-only inference | 20 GB+ VRAM is a practical starting point for full GPU offload at modest context; 16 GB GPUs need partial CPU offload |
| Text + image inference | 24 GB+ VRAM recommended; a 20 GB GPU can run modest contexts, while 16 GB GPUs need partial CPU offload |
| Long context | Extra memory is required for the KV cache; begin at 4K context on a 20 GB GPU and increase only after measuring memory use |
Actual memory use depends on context size, batch size, GPU offload layers, and runtime build. The model weights alone are about 15.4 GiB and the F16 vision projector adds about 0.8 GiB. A 20 GB GPU is a good starting point for 4K context, but does not make the full 128K-token capability practical without substantial CPU/RAM offload.
hf download unofficial-jona/MedGemma-27B-IT-GGUF \
MedGemma-27B-IT-Q4_K_M.gguf mmproj-MedGemma-27B-IT-F16.gguf \
--local-dir ./MedGemma-27B-IT-GGUF
llama-cli -m ./MedGemma-27B-IT-Q4_K_M.gguf \
-ngl 999 -c 8192 \
-p "<start_of_turn>user\nExplain the difference between bacterial and viral pneumonia for a clinician.<end_of_turn>\n<start_of_turn>model\n"
llama-mtmd-cli -m ./MedGemma-27B-IT-Q4_K_M.gguf \
--mmproj ./mmproj-MedGemma-27B-IT-F16.gguf \
--image ./image.png -ngl 999 -c 8192 \
-p "<start_of_turn>user\nDescribe the image cautiously and list any uncertainty.<end_of_turn>\n<start_of_turn>model\n"
Use the chat template supplied by your runtime when it offers a Gemma 3 / MedGemma template. Command-line option names can differ slightly across llama.cpp releases; run the executable with --help to confirm the multimodal image flag.
MedGemma is a research/developer model, not a medical device. It can make incorrect, incomplete, biased, or unsafe statements. Do not use it as the sole basis for diagnosis, triage, treatment, or other clinical decisions. Keep qualified clinical review and appropriate safeguards in any healthcare workflow.
Q4_K_M (main model); F16 vision projectorThis conversion does not change the original model's license, usage terms, or safety limitations. Access to the base model requires agreeing to its original terms on Hugging Face.
@article{sellergren2025medgemma,
title={MedGemma Technical Report},
author={Sellergren, Andrew and others},
journal={arXiv preprint arXiv:2507.05201},
year={2025}
}