phi3-mini-mental-health-qlora

Phi-3 Mini - Mental Health Text Classification (QLoRA)

A LoRA adapter for microsoft/Phi-3-mini-4k-instruct, QLoRA-fine-tuned to classify a short first-person statement into Normal / Depression / Stress.

⚠️ Research and education only - NOT a diagnostic, screening, or crisis tool. Labels are derived from social-media context, not clinical diagnoses, and the model must not be used to make decisions about real people.

Results (balanced held-out test set, 300 per class)

Per-class F1 (fine-tuned): Normal 0.91 · Depression 0.93 · Stress 0.95.

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16,
                         bnb_4bit_use_double_quant=True)
tok = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
base = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct", quantization_config=bnb,
                                            device_map="auto")
model = PeftModel.from_pretrained(base, "bs01338/phi3-mini-mental-health-qlora")

msgs = [
    {"role": "system", "content": "You classify a statement into one of: Normal, Depression, Stress. Respond with only the label."},
    {"role": "user", "content": "deadlines piling up and I can't keep up"},
]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
enc = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**enc, max_new_tokens=6)
print(tok.decode(out[0][enc.input_ids.shape[1]:], skip_special_tokens=True))

Training

QLoRA (4-bit NF4) · LoRA r=16, alpha=32 · 3 epochs · class-balanced 3-class subset (~2,137 per class) of the "Sentiment Analysis for Mental Health" dataset.

Full code: https://github.com/BilalAhmadSami/Mental-Health-Text-Classification-QLoRA