deberta-v3-base-emotion-classifier

Multilabel Emotion Classification Model (DeBERTa-v3-base)

Model Description

This model is fine-tuned DeBERTa-v3-base for multilabel emotion classification. It can predict multiple emotions simultaneously from text with superior performance using disentangled attention mechanisms.

Emotions Detected

amusement, anger, annoyance, caring, confusion, disappointment, disgust, embarrassment, excitement, fear, gratitude, joy, love, sadness

Performance

• Macro F1 Score: 0.3913
• Training Data: 37164 samples
• Validation Data: 9291 samples

Key Features

• Disentangled Attention: Separates content and position representations
• Enhanced Mask Decoder: Better handling of masked tokens
• Relative Position Bias: Improved positional understanding
• Multilabel Capability: Simultaneous prediction of multiple emotions

Usage

from transformers import AutoTokenizer, AutoModel
import torch

tokenizer = AutoTokenizer.from_pretrained("your-username/emotion-classifier-deberta")
model = AutoModel.from_pretrained("your-username/emotion-classifier-deberta")

# Example usage
text = "I'm so happy and excited about this!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=512)
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.sigmoid(outputs.logits)

Training Details

• Base Model: microsoft/deberta-v3-base
• Training Epochs: 2
• Learning Rate: 1e-05
• Batch Size: 16
• Max Length: 128
• Memory Optimizations: Gradient accumulation, FP16, gradient checkpointing

Model Architecture

• Total Parameters: 183,842,318
• Trainable Parameters: 183,842,318

Training Optimizations

• Mixed precision training (FP16)
• Gradient accumulation for memory efficiency
• Gradient checkpointing
• Early stopping based on macro F1 score