on-device-LLM-gpt-oss-20b

<div align="center">

Benchmarking and Adapting On-Device Large Language Models for Clinical Decision Support

</div> <div align="center"> <table align="center"> <tr> <td><a href="[arXiv link - to be added]" target="_blank"><img src="https://img.shields.io/badge/arXiv-Paper-FF6B6B?style=for-the-badge&logo=arxiv&logoColor=white" alt="Paper"></a></td> <td><a href="https://github.com/bowang-lab/on-device-LLM" target="_blank"><img src="https://img.shields.io/badge/GitHub-Code-181717?style=for-the-badge&logo=github&logoColor=white" alt="Code"></a></td> <td><a href="https://huggingface.co/wanglab/on-device-LLM-gpt-oss-20b" target="_blank"><img src="https://img.shields.io/badge/HuggingFace-Model-FFBF00?style=for-the-badge&logo=huggingface&logoColor=white" alt="HuggingFace Model"></a></td> <td><a href="https://huggingface.co/datasets/wanglab/eurorad-gpt-oss-training-data" target="_blank"><img src="https://img.shields.io/badge/HuggingFace-Dataset-28A745?style=for-the-badge&logo=huggingface&logoColor=white" alt="Dataset"></a></td> </tr> </table> </div>

Authors

<p align="center"> <a href="https://huggingface.co/alif-munim">Alif Munim</a>^{* 1}, <a href="https://scholar.google.com.hk/citations?hl=en&user=bW1UV4IAAAAJ">Jun Ma</a>^{* 1,2}, <a href="https://huggingface.co/omareng">Omar Ibrahim</a>^{* 1}, <b>Alhusain Abdalla</b>^{* 1}, Shuolin Yin³, Leo Chen⁴, <a href="https://scholar.google.ca/citations?user=37FDILIAAAAJ&hl=en">Bo Wang</a>^{† 1,5,6,7,8} </p> <p align="center"> ^* Equal contribution &nbsp;&nbsp;&nbsp; ^† Corresponding author </p> <p align="center"> ¹AI Collaborative Centre, University Health Network, Toronto, Canada<br> ²Princess Margaret Cancer Centre, University Health Network, Toronto, Canada<br> ³Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada<br> ⁴Division of Urology, Department of Surgery, St. Michael's Hospital, Unity Health Toronto and University of Toronto, Toronto, Canada<br> ⁵Peter Munk Cardiac Centre, University Health Network, Toronto, Canada<br> ⁶Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada<br> ⁷Department of Computer Science, University of Toronto, Toronto, Canada<br> ⁸Vector Institute for Artificial Intelligence, Toronto, Canada </p>

Highlights

• LoRA fine-tuned GPT-OSS 20B for structured radiology differential diagnosis
• Trained on 1,894 EuroRad medical cases spanning diverse imaging modalities and specialties
• Generates systematic chain-of-thought reasoning: symptom mapping → differential analysis → diagnosis
• Lightweight adapter (2.27 GB) compatible with 4-bit quantization for on-device deployment
• Part of a broader benchmark study comparing on-device LLMs across medical tasks

Model Overview

This model is a LoRA fine-tuned version of unsloth/gpt-oss-20b for medical radiology diagnosis, developed as part of a study benchmarking and adapting on-device large language models for clinical decision support. Trained on EuroRad clinical cases, it generates step-by-step diagnostic reasoning from patient history and imaging findings, mapping symptoms to differentials and converging on a final diagnosis with supporting evidence.

The model employs a systematic diagnostic framework: (1) relating clinical history to imaging findings, (2) mapping findings to each differential, (3) systematic elimination of alternatives, and (4) converging on a final diagnosis with confidence reasoning.

Model Details

Installation

pip install unsloth peft transformers accelerate bitsandbytes

Usage

from unsloth import FastLanguageModel
from peft import PeftModel

# Load base model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/gpt-oss-20b",
    dtype=None,
    max_seq_length=4096,
    load_in_4bit=True,
    full_finetuning=False,
)

# Load LoRA adapter
model = PeftModel.from_pretrained(
    model,
    "wanglab/on-device-LLM-gpt-oss-20b",
    is_trainable=False
)

# Enable inference mode
FastLanguageModel.for_inference(model)

# Example inference
prompt = """You are an expert radiologist demonstrating step-by-step diagnostic reasoning.

Case presentation:
{combined_description}

Differential diagnoses to consider:
{dd_formatted}

Generate systematic Chain-of-Thought reasoning that shows how clinicians think through cases:
1. **Connect symptoms to findings**: Link clinical presentation with imaging observations
2. **Map to differentials**: Show how findings support or contradict each differential diagnosis
3. **Systematic elimination**: Explicitly rule out less likely options with reasoning
4. **Converge to answer**: Demonstrate the logical path to the correct diagnosis"""

inputs = tokenizer(prompt.format(
    combined_description="...",  # clinical history + imaging findings
    dd_formatted="Diagnosis A, Diagnosis B, Diagnosis C"
), return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Citation

Citation will be updated upon arXiv submission and journal publication.

@article{munim2025ondevice,
    title={Benchmarking and Adapting On-Device Large Language Models for Clinical Decision Support},
    author={Munim, Alif and Ma, Jun and Ibrahim, Omar and Abdalla, Alhusain and Yin, Shuolin and Chen, Leo and Wang, Bo},
    journal={},
    year={2025}
}

Limitations

• Clinical Validation Required: This model has not been clinically validated and should not be used for actual patient diagnosis
• Research Purposes Only: Designed for research in medical AI and diagnostic systems
• Not for Clinical Use: Not intended for direct patient care without clinical validation
• May reflect biases present in the EuroRad training data
• Performance may vary across imaging modalities and medical specialties
• Like all LLMs, may generate plausible but incorrect information ("hallucinations")

Contact

For issues and questions, please open a discussion in this repository.
Corresponding author: Bo Wang — [email protected]

Disclaimer: This model is for research purposes only and has not been approved for clinical use. Always consult qualified healthcare professionals for medical decisions.