Downloads · 30 days
0
k298976/medgemma-1.5-4b-it-nodulocc-cls-qlora-adapter
medgemma-1.5-4b-it-nodulocc-cls-qlora-adapter is a image-text-to-text model from k298976. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as other.
QLoRA adapter fine-tuned on top of google/medgemma-1.5-4b-it for binary lung nodule classification from frontal chest X-rays. Developed as part of the NoduLoCC 2026 Challenge (Team MICLAB, UNICAMP).
Downloads · 30 days
0
Access
Public
Updated Mar 13, 2026
Repo size
188 MB
Likes
0
Public
Click a slice to open those files.
.safetensors154 MB · 82%
From the Hugging Face model README
QLoRA adapter fine-tuned on top of google/medgemma-1.5-4b-it for binary lung nodule classification from frontal chest X-rays. Developed as part of the NoduLoCC 2026 Challenge (Team MICLAB, UNICAMP).
Link to Github with training and inference scripts..
⚠️ License requirement: The base model is gated. You must have a Hugging Face account, acknowledge the Health AI Developer Foundations license, and supply a
HF_TOKENwith read permissions to use this adapter.
google/medgemma-1.5-4b-itBinary lung nodule detection from frontal chest X-rays. The model answers with a single letter token (A = Healthy, B = Nodule present) and the predicted class is derived from the relative log-probabilities of those two tokens at the final input position. See the inference script below.
transformers, peft, bitsandbytes, Pillow, python-dotenvHF_TOKEN in your environment (or .env file)pip install transformers peft bitsandbytes accelerate pillow python-dotenv tqdm pandas
A complete, ready-to-run inference script is available in the GitHub repository. Minimal example:
import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
BASE_MODEL = "google/medgemma-1.5-4b-it"
ADAPTER_ID = "k298976/medgemma-1.5-4b-it-nodulocc-cls-qlora-adapter"
model = AutoModelForImageTextToText.from_pretrained(
BASE_MODEL,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(model, ADAPTER_ID)
processor = AutoProcessor.from_pretrained(BASE_MODEL)
model.eval()
PROMPT = (
"You are a radiology assistant. Determine whether this frontal chest X-ray "
"shows a lung nodule.\n"
"Answer with exactly one letter:\n"
"A = Healthy (no lung nodule)\n"
"B = Nodule present"
)
messages = [
{
"role": "user",
"content": [
{"type": "image"},
{"type": "text", "text": PROMPT},
],
}
]
image = Image.open("your_xray.png").convert("RGB")
prompt_text = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=False
)
inputs = processor(
text=[prompt_text], images=[[image]], return_tensors="pt"
).to(model.device)
a_id = processor.tokenizer.encode("A", add_special_tokens=False)[0]
b_id = processor.tokenizer.encode("B", add_special_tokens=False)[0]
with torch.inference_mode():
logits = model(**inputs).logits # (1, seq_len, vocab)
last_logits = logits[0, -1, :] # last input position
ab_probs = torch.softmax(last_logits[[a_id, b_id]].float(), dim=-1)
p_nodule = ab_probs[1].item()
label = "Nodule" if p_nodule >= 0.5 else "No Finding"
print(f"Prediction: {label} | p(Nodule) = {p_nodule:.4f}")
Image preprocessing: All images should be percentile-clipped to [0.5, 95.5] and rescaled to uint8 before being passed to the processor — this matches training preprocessing. The full
load_image_for_model()helper is in the inference script.
The adapter was trained on the NoduLoCC 2026 Challenge dataset, which combines:
The dataset is heavily imbalanced (~95% healthy, ~5% nodule). The split used was 80% training / 5% validation / 15% test, stratified by class.
All input images are loaded in their native bit depth, converted to single-channel grayscale, then percentile-clipped to [0.5, 95.5] and rescaled to uint8. The resulting grayscale image is converted to RGB before being passed to the MedGemma processor (which normalizes to 896×896).
The base model weights are quantized to 4-bit NF4 with double quantization. Only a small set of low-rank adapter weights are trained, keeping most of the base model frozen.
Quantization config:
BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_quant_storage=torch.bfloat16,
)
LoRA config:
LoraConfig(
lora_alpha=16,
lora_dropout=0.05,
r=16,
bias="none",
target_modules="all-linear",
task_type="CAUSAL_LM",
)
| Hyperparameter | Value |
|---|---|
| Training regime | bf16 mixed precision |
| Optimizer | AdamW (fused) |
| Peak learning rate | 2×10⁻⁴ |
| LR scheduler | Linear warmup (3% steps) + linear decay |
| Effective batch size | 16 (batch 4 × grad accum 4) |
| Total steps | 4400 |
Performance of the fine-tuned adapter vs. zero-shot MedGemma 1.5 on the held-out test set (decision threshold = 0.5):
| Model | Precision | Recall | Specificity | F1 | ROC AUC | PR AUC |
|---|---|---|---|---|---|---|
| MedGemma 1.5 Zero-shot | 0.1087 | 0.1478 | 0.9457 | 0.1253 | 0.6913 | 0.0850 |
| MedGemma 1.5 + this adapter | 0.374 | 0.3621 | 0.9728 | 0.368 | 0.7864 | 0.2384 |
Fine-tuning more than doubled the F1 score and produced substantial gains in ROC AUC and PR AUC. Sensitivity remains a challenge and threshold tuning on a held-out validation set is recommended for clinical adaptation.
If you use this adapter, please cite the NoduLoCC 2026 Challenge and the MedGemma technical report:
@article{sellergren2025medgemma,
title={MedGemma Technical Report},
author={Sellergren, Andrew and Kazemzadeh, Sahar and others},
journal={arXiv preprint arXiv:2507.05201},
year={2025}
}
Kristhian André Oliveira Aguilar, Diedre Santos do Carmo, Letícia Rittner — MICLab, UNICAMP