medgemma-histolab-5k

HistoLab — MedGemma-4B Fine-tuned for Histopathology

A QLoRA fine-tuned adapter on top of google/medgemma-4b-it, trained to classify histopathology patches across three benchmark datasets.

Results

Model Details

• Base model: google/medgemma-4b-it (4B parameter multimodal LLM)
• Fine-tuning method: QLoRA (4-bit base + bfloat16 adapter merge at inference)
• LoRA config: r=16, α=32, dropout=0.05, target=language layers only
• Training: 1 epoch, lr=2e-5, batch_size=1 + 4 grad accumulation steps
• Adapter size: ~35 MB

Training Data

Usage

This repo contains only the LoRA adapter weights (~35 MB). The base model (google/medgemma-4b-it) is downloaded separately and requires a HuggingFace token with access to the gated MedGemma model.

Quickest way — Kaggle demo notebook

The notebook handles everything: GPU setup, model download, adapter loading, and launches a public Gradio UI.

Load with the HistoLab app

git clone https://github.com/karaditya/medgemma-histolab
cd medgemma-histolab
pip install -e .
export HF_TOKEN=hf_...
export ADAPTER_REPO_ID=karadi97/medgemma-histolab-5k
python app.py

Load manually with PEFT

from transformers import AutoProcessor, AutoModelForImageTextToText
from peft import PeftModel
import torch

BASE = "google/medgemma-4b-it"
ADAPTER = "karadi97/medgemma-histolab-5k"
TOKEN = "hf_..."   # needs MedGemma access

processor = AutoProcessor.from_pretrained(BASE, token=TOKEN)
base = AutoModelForImageTextToText.from_pretrained(
    BASE, torch_dtype=torch.bfloat16, device_map="auto", token=TOKEN
)
model = PeftModel.from_pretrained(base, ADAPTER, token=TOKEN)
model = model.merge_and_unload()   # optional: bake adapter in for faster inference

Intended Use & Limitations

• Intended for: Research, education, and demonstration purposes.
• Not intended for: Clinical diagnosis or patient care decisions.
• The model was trained on patch-level images (224×224 px) — whole-slide inference requires tiling.
• BACH accuracy (68.3%) is limited by the small dataset size (~400 images total).

License

The adapter weights are released under Apache 2.0. The base model (google/medgemma-4b-it) is subject to Google's MedGemma Terms of Use.