empathetic-deberta-v3

Empathetic-Deberta-v3

Model type: microsoft/deberta-v3-base fine-tuned for emotion classification Repository: https://huggingface.co/mcgowee/empathetic-deberta-v3 License: Apache-2.0

Model Overview

Empathetic-Deberta-v3 is a transformer-based sequence classification model that identifies the underlying emotion of a user’s input, leveraging DeBERTa-v3’s advanced self-attention mechanisms for robust contextual understanding fileciteturn2file0.

Dataset

• Source dataset: empathetic_dialogues_v2 from Hugging Face (Adapting et al., 2023) fileciteturn2file3.
• Source field: chat_history, split into individual user utterances
• Target field: emotion, labeling each utterance with one of 32 emotion categories

Training Details

• Architecture:

  • Task: Sequence Classification
  • Base model: microsoft/deberta-v3-base

• Hyperparameters:

  • Learning rate: 3e-5
  • Number of epochs: 10
  • Weight decay: 0.01

• Libraries & Tools:

  • PyTorch for model training
  • scikit-learn for label encoding and train-test splitting

• Preprocessing:

  • Extract individual user turns from chat_history
  • Encode emotion labels via LabelEncoder

Usage Example

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("mcgowee/empathetic-deberta-v3")
model = AutoModelForSequenceClassification.from_pretrained("mcgowee/empathetic-deberta-v3")

# Create classification pipeline
classifier = pipeline(
    "text-classification",
    model=model,
    tokenizer=tokenizer,
    device=0  # use -1 for CPU
)

# Example inference
text = "I’m frustrated that my flight got canceled again."
result = classifier(text)
print(result)

Intended Uses and Limitations

Intended uses:

• Emotion detection in chatbots and conversational agents
• Preprocessing step for downstream empathetic response generation

Limitations & Risks:

• Not a substitute for professional emotional support
• Performance depends on representativeness of training data; may misclassify less common emotions
• Requires evaluation for bias and fairness in sensitive applications

Citation

@misc{mcgowee_empathetic-deberta-v3_2025,
  title = {Empathetic-Deberta-v3: DeBERTa-v3 Model Fine-Tuned for Emotion Classification},
  author = {Earl McGowen (mcgowee)},
  year = {2025},
  howpublished = {\url{https://huggingface.co/mcgowee/empathetic-deberta-v3}}
}

References

• Adapting et al. (2023). empathetic_dialogues_v2 [Data set]. Hugging Face. fileciteturn2file3
• He, P., Liu, X., Gao, J., & Chen, W. (2021). DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2111.09543 fileciteturn2file5