llava-rad-iu-v3

llava-rad-iu-v3 (checkpoint-531)

LoRA adapter weights for the v3 run of microsoft/LLaVA-Rad (vicuna-7b-v1.5 + frozen BiomedCLIP-CXR 518 vision tower), fine-tuned on 2837 open-IU chest X-ray image-report pairs.

• training: full bf16 LoRA, r=64 / alpha=128 / dropout 0.05, lr 1e-4 cosine with 3% warmup, effective batch 16, 3 epochs (checkpoint at step 531), 2x RTX 5060 Ti 16 GB, DeepSpeed ZeRO-3 + CPU offload
• result on the held-out IU test set (n=500), vs official un-fine-tuned baseline: RadGraph rg_er 0.3503 vs 0.2877, ROUGE-L 0.2794 vs 0.2037

Contents

• LoRA adapter weights + adapter_config.json
• non_lora_trainables.bin - the re-trained mm_projector (4 keys); it must be installed on top of the base model before loading the adapter, which the loaders in the code repo do for you
• config.json - the LLaVA model config used at load time

How to load

Do not merge the adapter and do not quantize. Use the bf16 loaders from the code repo (https://github.com/vijayakumarharsath/llava-rad-iu):

• run_v3_eval.py (IU eval) or mimic_eval.py (MIMIC eval) both implement the full load path: vicuna-7b-v1.5 base + config patch + projector install from non_lora_trainables.bin + PeftModel.from_pretrained (unmerged), all tensors in bf16 to match training.

Notes

• trained only on the public open-IU collection; MIMIC-CXR was used for external validation only, never for training
• research software, produces draft reports for human review, not a diagnosis