multimodal-clinical-reasoning

<img src="./assets/radiology-header.svg" width="100%" alt="Multimodal Clinical Reasoning research card">

Multimodal Clinical Reasoning for Chest X-Ray Reports

Research card — no model weights or clinical system are released here.
This repository documents an academic deep-learning project and links to its reproducible notebooks, evaluation artifacts and report.

Status at a glance

Project overview

The project studies image-conditioned radiology report generation with a frozen medical vision encoder and a language decoder. Cross-attention connects chest X-ray patch representations to generated report tokens.

Chest X-ray image → BiomedCLIP vision encoder → patch features
                                                    │
                                                    ▼
Report tokens → GPT-2 decoder with cross-attention → generated report

Four controlled variants examine whether attention regularization changes generation quality and alignment behavior:

Evaluation snapshot

The companion repository reports held-out evaluation on 1,000 samples.

The regularized variants did not consistently outperform the baseline across text-generation metrics. The entropy-only variant produced the highest reported finding-overlap score, illustrating why multiple evaluation dimensions are necessary for medical report generation.

Contribution and attribution

This is a collaborative academic project. The source repository's recorded contribution history attributes the following work to Arda Çiftçi (arxp / OxyOxygen):

• VQA-RAD data cleaning and exploratory/statistical analysis;
• engineering 34 text, image and clinical features across 2,244 cleaned examples;
• dataset documentation and research-summary artifacts.

The public project later evolved into the MIMIC-CXR report-generation and ablation pipeline summarized above. This card distinguishes Arda's recorded contribution from the complete team's final system.

Intended use

This card is intended for:

• inspecting an academic multimodal deep-learning workflow;
• reviewing an ablation-study design;
• understanding the limits of automated radiology report generation;
• navigating to the reproducible notebooks and project report.

It is not an inference endpoint, diagnostic device, clinical decision-support system or substitute for a qualified radiologist.

Limitations and risks

• No clinical validation, prospective study or regulatory review was performed.
• Generated text may omit, hallucinate or misstate clinically important findings.
• BLEU, METEOR, ROUGE and CIDEr do not fully measure clinical correctness.
• Dataset composition and institutional practices may limit generalization.
• MIMIC-CXR access and redistribution remain subject to the dataset provider's terms.
• Checkpoints and source medical data are intentionally excluded from this Hugging Face repository.

Reproducibility resources

• Source repository and notebooks
• Project report
• Arda Çiftçi on GitHub
• Istanbul Mobility & Mood Explorer Space

Citation

If you reference this educational artifact, cite the companion GitHub repository and its contributors. This Hugging Face page is a research summary, not a released model package.