medical-deepseek-reasoning-8b

Medical DeepSeek Reasoning Model

This model is a fine-tuned version of DeepSeek-R1-Distill-Llama-8B on medical chain-of-thought reasoning data.

Model Description

• Base Model: DeepSeek-R1-Distill-Llama-8B
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Dataset: Medical O1 Reasoning SFT Dataset
• Training: 500 samples, 1 epoch
• Specialization: Medical reasoning with step-by-step chain-of-thought

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "Soumyajit-7/medical-deepseek-reasoning-8b",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Soumyajit-7/medical-deepseek-reasoning-8b")

# Example usage
prompt = '''Below is an instruction that describes a task, paired with an input that provides further context. 
Write a response that appropriately completes the request. 
Before answering, think carefully about the question and create a step-by-step chain of thoughts to ensure a logical and accurate response.

### Instruction:
You are a medical expert with advanced knowledge in clinical reasoning, diagnostics, and treatment planning. 
Please answer the following medical question. 

### Question:
A 61-year-old woman with a long history of involuntary urine loss during activities like coughing or sneezing but no leakage at night undergoes a gynecological exam and Q-tip test. Based on these findings, what would cystometry most likely reveal about her residual volume and detrusor contractions?

### Response:
<think>'''

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=1200,
    temperature=0.7,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

• Training Framework: Unsloth
• Optimization: LoRA with rank 16
• Batch Size: 2 per device, gradient accumulation steps: 4
• Learning Rate: 2e-4
• Max Sequence Length: 2048
• Quantization: 4-bit

Intended Use

This model is designed for educational and research purposes in medical reasoning. It should not be used for actual medical diagnosis or treatment decisions.

Limitations

• Limited to 500 training samples
• Should not replace professional medical advice
• May generate incorrect medical information
• Requires careful validation for any medical applications