Downloads · 30 days
117
23% of all-time downloads
ShahzebKhoso/qwen2.5-instruct-0.5B-pubmedqa-lora
qwen2.5-instruct-0.5B-pubmedqa-lora is a text generation model from ShahzebKhoso. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This model is a LoRA-adapted version of Qwen2.5-0.5B-Instruct, fine-tuned using Unsloth on the pqalabeled subset of the PubMedQA dataset.
Downloads · 30 days
117
23% of all-time downloads
All-time downloads
512
Public
Repo size
18.5 MB
Likes
0
Public
Click a slice to open those files.
.json14.2 MB · 70%
From the Hugging Face model README
This model is a LoRA-adapted version of Qwen2.5-0.5B-Instruct, fine-tuned using Unsloth on the pqa_labeled subset of the PubMedQA dataset.
This work demonstrates that even a compact instruction-tuned model like Qwen2.5 0.5B Instruct can achieve near state-of-the-art performance on biomedical QA tasks. With LoRA fine-tuning using just 1,000 examples, this model achieves 98.99% accuracy on the PubMedQA test set.
It reframes the classification task as a text generation problem — prompting the model to generate "yes", "no", or "maybe" responses. This results in highly interpretable and efficient predictions with excellent generalization.
| Label | Precision | Recall | F1 Score | Support |
|---|---|---|---|---|
| yes | 0.981 | 1.000 | 0.990 | 52 |
| no | 1.000 | 1.000 | 1.000 | 38 |
| maybe | 1.000 | 0.889 | 0.941 | 9 |
r: 16alpha: 16target_modules: ["q_proj", "v_proj"]from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(model, "ShahzebKhoso/qwen2.5-instruct-0.5B-pubmedqa-lora")
tokenizer = AutoTokenizer.from_pretrained("ShahzebKhoso/qwen2.5-instruct-0.5B-pubmedqa-lora")
@misc{shahzebkhoso2025qwenpubmedqa,
title={Fine-tuning Qwen2.5-0.5B on PubMedQA with LoRA},
author={Shahzeb Khoso},
year={2025},
howpublished={\\url{https://huggingface.co/ShahzebKhoso/qwen2.5-instruct-0.5B-pubmedqa-lora}},
}