kluebert-mental-health-severity

KLUE-BERT Mental Health Severity Models

Three fine-tuned klue/bert-base models that each read a Korean counseling-dialogue transcript and output a severity score for one condition:

Each model is a regression head on top of KLUE-BERT: it takes the [CLS] token's hidden state and outputs a single scalar, trained (via MSE loss) against a 0-3 integer severity label. Round and clip the raw output to [0, 3] to recover the discrete label used at training time.

Private / internal model. Trained on counseling-dialogue transcripts for a research project; not evaluated for clinical or diagnostic use. See "Limitations" below before using outputs for anything beyond research.

Architecture note

This is not a stock BertForSequenceClassification. The regression head reads the raw [CLS] hidden state directly (no BertPooler), so it needs the custom CustomBertForSequenceRegression class shipped in each subfolder (modeling_kluebert_regression.py). Load with trust_remote_code=True.

Usage

import torch
from transformers import AutoModel, AutoTokenizer

repo_id = "sso5803/kluebert-mental-health-severity"
condition = "depression"  # or "addiction" / "anxiety"

model = AutoModel.from_pretrained(repo_id, subfolder=condition, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder=condition)
model.eval()

text = "요즘 계속 잠도 안 오고 기운이 없어."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
    score = model(**inputs).item()

severity = min(max(round(score), 0), 3)
print(f"raw score: {score:.3f}, severity (0-3): {severity}")

Training

• Base model: klue/bert-base
• Objective: MSE regression against a 0-3 severity label per condition
• One model trained independently per condition (addiction, anxiety, depression) on transcripts labeled for that condition plus normal (non-condition) examples
• 100 epochs, lr 2e-5, batch size 16, weight decay 0.01
• Original Colab training/inference scripts (kluebert_train.py, kluebert_run.py) are not included in this repo.

Limitations

• Trained on a specific counseling-transcript dataset; may not generalize to other text domains, conversational styles, or age groups.
• Not validated as a clinical or diagnostic tool. Do not use for real-world screening or treatment decisions without proper clinical validation.
• Each condition model was trained independently; scores across addiction/anxiety/depression are not necessarily on a comparable scale.