PsychoClass

Mental State Text Classification with DistilBERT

This model is a fine-tuned version of distilbert/distilbert-base-multilingual-cased for multiclass classification of short English statements into seven mental-state categories.

It was developed as part of a 2026 master's thesis on language modelling and fine-tuning language models for mental-state text classification.

Important: This is a research model, not a medical or psychological diagnostic system. Its output must not be interpreted as a clinical diagnosis, risk assessment, or substitute for a qualified professional.

Model description

• Task: Multiclass text classification
• Base model: distilbert/distilbert-base-multilingual-cased
• Architecture: DistilBERT with a sequence-classification head
• Number of classes: 7
• Recommended input: One sentence or a short statement
• Maximum training sequence length: 256 tokens
• Framework: PyTorch and Hugging Face Transformers

The base model contains six Transformer layers, a hidden size of 768, 12 attention heads, and approximately 134 million parameters. It was selected to provide a practical balance between classification performance and computational requirements.

Labels

The predicted categories describe patterns learned from the dataset. They do not establish whether a person has any medical condition.

Intended use

The model is intended for:

• research on multiclass text classification;
• educational demonstrations of fine-tuning DistilBERT;
• exploratory analysis of short statements;
• prototyping systems that require human review of model predictions.

The model is not intended for:

• clinical diagnosis or treatment decisions;
• emergency or suicide-risk assessment;
• autonomous moderation or decisions affecting access to care, employment, education, insurance, or other high-impact services;
• surveillance or profiling of individuals;
• use without appropriate privacy protections and informed consent.

Training data

The model was fine-tuned on the Kaggle Sentiment Analysis for Mental Health dataset. The dataset contains 51,074 labelled statements collected from several public online sources, including social-media platforms, forums, Twitter/X, and Reddit.

The class distribution reported in the thesis is imbalanced:

Because the dataset aggregates multiple sources, its examples may differ in writing style, length, subject matter, annotation methods, and annotation reliability.

The data was randomly divided using seed 42 into approximately:

• 80% training data;
• 10% validation data;
• 10% test data.

Training procedure

Logging, validation, and checkpoint saving were performed after every epoch. Checkpoint selection was configured to maximize Macro F1.

Evaluation

Macro F1 was the main evaluation metric because the dataset is imbalanced. The fine-tuned model reached a test Macro F1 of 0.803, compared with 0.096 for the unfine-tuned base model in the reported experiment.

Test F1 by class

The most important reported confusion was between the Suicidal and Depression categories. The model also sometimes classified Stress examples as Normal.

Limitations and risks

• The dataset is class-imbalanced, and the least frequent classes may be less reliably represented.
• The dataset combines several online sources with potentially inconsistent annotation standards.
• Labels may reflect subjective or noisy annotations rather than clinical assessments.
• The model may learn unintended correlations with text length, vocabulary, platform, or writing style.
• Removing punctuation may discard useful emotional or semantic information.
• The model was designed for individual sentences and short statements. Long documents are truncated and may require sentence-level processing and aggregation.
• The model has difficulty distinguishing semantically related categories, especially Depression and Suicidal.
• Performance may decrease on languages, communities, platforms, and writing styles that differ from the training data.
• A high softmax score is model confidence, not proof that the predicted category is correct.

@mastersthesis{butera2026mentalstate,
  author = {Butera, Gaetano Antonio},
  title = {Language Modelling and Fine-Tuning Large Language Models for the Classification of Texts by Mental-State Category},
  school = {Volga State University of Technology},
  year = {2026}
}