Medic-chatbot

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This project focuses on building a medical domain-specific chatbot using the DeepSeek R1 model. The chatbot is fine-tuned on the Medical O1 Reasoning SFT dataset to provide accurate, step-by-step reasoning for medical queries. It is designed to assist healthcare professionals, students, and patients by offering quick, reliable answers to medical questions while ensuring domain specificity.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

Purpose of the Chatbot The purpose of this project is to develop a medical domain-specific chatbot capable of understanding and responding to medical queries with logical, step-by-step reasoning. The chatbot is designed to assist healthcare professionals, students, and patients by providing accurate and concise medical information. Domain Alignment The chatbot is aligned with the medical domain, focusing on clinical reasoning, diagnostics, and treatment planning. It is trained on the Medical O1 Reasoning SFT dataset, which contains medical questions, chain-of-thought reasoning, and responses. This alignment ensures that the chatbot can handle complex medical queries while maintaining domain specificity. Relevance and Necessity

1. Improved Access to Medical Knowledge: The chatbot provides quick, reliable answers to medical questions, reducing the need for extensive manual research.
2. Assistance for Healthcare Professionals: It helps healthcare professionals by handling routine inquiries, allowing them to focus on more critical tasks.
3. Educational Tool: The chatbot serves as a learning aid for medical students, offering detailed explanations for medical concepts.

• Developed by: Oche Ankeli
• Funded by : Self-funded

Hardware Type

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Hardware Type: GPU (A100 or T4) is sufficient

Preprocessing

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

Preprocessing Steps

1. Tokenization: The dataset uses the DeepSeek R1 tokenizer, which employs WordPiece for subword tokenization.
2. Normalization: ○ Removed noise (e.g., special characters, irrelevant text). ○ Handled missing values by dropping rows with incomplete data. ○ Ensured consistent formatting across the dataset.
3. Formatting: ○ Structured the dataset to include a system prompt, question, chain-of-thought reasoning, and response. ○ Added a domain-specific restriction for non-medical questions.

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

The fine-tuned model was evaluated using the following metrics:

Quantitative Metrics BLEU Score: Measures the similarity between the model's responses and reference answers.

Initial Model: 0.70 Fine-Tuned Model: 0.90 Improvement: 28.57% Perplexity: Measures how well the model predicts the next token.

Initial Model: 3.50 Fine-Tuned Model: 2.68 Improvement: 23.43% BERTScore: Evaluates the precision, recall, and F1 score of the generated text.

Precision: 0.80 (14.29% improvement) Recall: 0.80 (23.08% improvement) F1: 0.80 (19.40% improvement)

Summary

The chatbot effectively handles medical queries while maintaining domain specificity, making it a valuable tool for healthcare professionals, students, and patients.

• PEFT 0.14.0