smollm2-medical-lora

Model Description

SmolLM2-Medical-LoRA is a fine-tuned version of SmolLM2-135M, optimized for short medical Q&A tasks. The model was trained using LoRA (Low-Rank Adaptation) for efficient and fast training on Google Colab.

Typical tasks:

Answering medical questions

Explaining symptoms

Providing general health information

Simplifying complex medical topics

Dataset

The dataset consists of instruction–response pairs created specifically for this project.

Dataset Format (JSONL)

Each sample follows: { "text": "<instruction>\n<answer>" }

Training details

Base Model: SmolLM2-135M

Method: LoRA fine-tuning

Epochs: 7

Learning Rate: 2e-4

Batch Size: 2 (with gradient accumulation)

Max Length: 256 tokens

GPU: Google Colab T4

Frameworks: HuggingFace Transformers, Accelerate, PEFT

How to Use this model

Load with Transformers: from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("OmarrKhaledd/smollm2-medical-lora") tokenizer = AutoTokenizer.from_pretrained("OmarrKhaledd/smollm2-medical-lora")

prompt = "What could cause chest pain and shortness of breath?" inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=150) print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example Output

User: “What causes pain when swallowing?”

Model: “Pain when swallowing (odynophagia) can happen due to throat infections, acid reflux, tonsillitis, or irritation of the esophagus…”

Evaluation

Model generates coherent, short medical explanations

Useful for educational Q&A

Safe: avoids giving diagnosis or prescriptions

Limitation

Not a real medical assistant

Limited dataset (small educational fine-tune)

May produce inaccurate or incomplete explanations

License

This model is released for research and educational use only. Not intended for clinical, diagnostic, or therapeutic applications..