swasthai-medical-7b

SwasthAI Medical 7B

A fine-tuned Mistral-7B model specialized for Indian healthcare assistance. SwasthAI provides accurate, culturally-appropriate medical information for Indian patients.

Model Details

Training Data

Fine-tuned on 1,223 curated medical Q&A pairs covering:

• General medicine and primary care
• Chronic disease management (diabetes, hypertension, thyroid)
• Maternal and child health
• Mental health and wellness
• Nutrition and lifestyle guidance
• Ayurveda and traditional Indian medicine
• Healthcare navigation in India
• Emergency care guidance

Quick Start

With Unsloth (Recommended - Fastest)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="sri2000us/swasthai-medical-7b",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

def ask_swasthai(question):
    prompt = f"""### Instruction:
You are SwasthAI, an expert Indian healthcare assistant. Answer the medical question accurately.

### Input:
{question}

### Response:
"""
    inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_new_tokens=256, use_cache=True)
    response = tokenizer.batch_decode(outputs)[0]
    return response.split("### Response:")[-1].strip()

# Example
print(ask_swasthai("What are the symptoms of diabetes?"))

With Transformers + PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.3",
    load_in_4bit=True,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.3")

# Load LoRA adapters
model = PeftModel.from_pretrained(base_model, "sri2000us/swasthai-medical-7b")

Prompt Format

This model uses the Alpaca prompt format:

### Instruction:
You are SwasthAI, an expert Indian healthcare assistant. Answer the medical question accurately.

### Input:
{user_question}

### Response:
{model_response}

Example Outputs

Q: What are the symptoms of Type 2 diabetes?

Common symptoms include increased thirst and frequent urination, unexplained weight loss, fatigue, blurred vision, slow-healing wounds, and numbness in extremities. If you experience these, consult a doctor for HbA1c and fasting blood sugar tests.

Q: How can I manage high blood pressure naturally?

Reduce sodium intake, follow DASH diet, exercise 30 minutes daily, maintain healthy weight, limit alcohol, practice yoga/meditation, and include potassium-rich foods. Always continue prescribed medications.

Limitations

• Not a replacement for professional medical advice - Always consult qualified healthcare providers
• Information currency - Medical knowledge evolves; verify critical information
• Emergency situations - Call 102/108 in India for medical emergencies
• Individual variation - Responses are general guidance

Training Configuration

# LoRA Configuration
r = 16
lora_alpha = 16
lora_dropout = 0
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]

# Training Arguments
learning_rate = 2e-4
batch_size = 2
gradient_accumulation_steps = 4
num_epochs = 1
optimizer = "adamw_8bit"

Environmental Impact

• Hardware: Google Colab T4 GPU (16GB VRAM)
• Training Time: ~25 minutes
• Cloud Provider: Google Colab

Citation

@misc{swasthai-medical-7b,
  author = {SwasthAI Team},
  title = {SwasthAI Medical 7B: Fine-tuned Mistral for Indian Healthcare},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/sri2000us/swasthai-medical-7b}
}

Acknowledgments

• Unsloth for efficient fine-tuning
• Mistral AI for the base model
• Hugging Face for model hosting

Framework versions

• PEFT 0.18.0
• Unsloth 2024.12
• TRL 0.12.0