deberta-multitask-sentiment-analysis

Model Card for DeBERTa Multitask Model

This model is a multitask text classification model based on DeBERTa V3 Base, designed to handle emotion detection, polarity classification, and hate speech detection simultaneously. It leverages a shared DeBERTa backbone with task-specific LSTMs, attention, and dense layers for each task. The model is fine-tuned on custom datasets for these tasks.

Model Details

Model Description

This multitask DeBERTa model performs three text classification tasks:

1. Emotion Detection: Identifies 8 distinct emotions.
2. Polarity Classification: Detects the sentiment polarity across 4 levels.
3. Hate Speech Detection: Detects whether the input text contains hate speech (binary classification).

• Developed by: Vivek Sham
• Language(s) (NLP): English
• Model type: Multitask DeBERTa Model
• Finetuned from model: microsoft/deberta-v3-base

Uses

Direct Use

This model can be used directly for text and Social Media Comments:

• Emotion detection: Analyze the emotional tone in text.
• Polarity classification: Detect sentiment polarity from negative to positive.
• Hate speech detection: Detect hate speech in text content.

Bias, Risks, and Limitations

This model may exhibit biases based on the nature of the dataset it was fine-tuned on. These biases could include cultural, demographic, or contextual factors. Users should consider that the model may not generalize well across different datasets, and it might show inaccuracies when applied to texts with sarcasm, slang, or specific cultural references.

How to Get Started with the Model

from transformers import DebertaV2Tokenizer, DebertaV2Model
import torch

# Load the tokenizer and model
tokenizer = DebertaV2Tokenizer.from_pretrained('path_to_model')
model = torch.load('path_to_model/pytorch_model.bin')

# Tokenize input text
inputs = tokenizer(["I love this!", "I hate this!"], return_tensors='pt', max_length=256, truncation=True, padding=True)

# Predict
with torch.no_grad():
    outputs = model(**inputs)

# Get logits for each task
emotion_logits = outputs['emotion']
polarity_logits = outputs['polarity']
hate_speech_logits = outputs['hate_speech']

Training Details

Training Data

The model was fine-tuned on custom datasets for emotion detection, polarity classification, and hate speech detection. The data includes text samples from social media, reviews, and other platforms relevant to these tasks.

Training Procedure

Data preprocessing involved tokenization using DeBERTa's tokenizer, padding to a maximum length of 256 tokens, and truncating where necessary.

Training Hyperparameters

Training Hyperparameters

• Batch size: 32
• Learning rate: 5e-5
• Epochs: 10
• Optimizer: AdamW

Results

The model achieved the following accuracy scores:

• Emotion Detection: 92%
• Polarity Classification: 95%
• Hate Speech Detection: 99%