SentimentAnalysis-distilbert-base-uncased-finetuned-emotion

SentimentAnalysis-distilbert-base-uncased-finetuned-emotion

This model is a fine-tuned version of distilbert-base-uncased for emotion classification of short texts (tweets).
It predicts one of six emotions:

• sadness
• joy
• love
• anger
• fear
• surprise

The model was trained using the 🤗 Transformers Trainer API with class-weighted loss to handle class imbalance.

---

Model performance

Evaluation results on the test set:

• Loss: 0.2094
• F1 Macro: 0.8902
• Accuracy: 0.927

Note: F1 Macro is the primary metric since the dataset is imbalanced.

---

Intended uses

This model is suitable for:

• Emotion analysis of tweets or short social media texts
• NLP research and educational projects
• Sentiment-aware chatbots or analytics dashboards

---

Limitations

• Trained on short texts (tweets); performance may degrade on long documents
• English-only
• May inherit biases present in the training data
• Not intended for high-stakes or sensitive decision-making

---

Training data

The model was trained on an emotion-labeled tweet dataset with six emotion classes.
The dataset was split into training, validation, and test sets.

Preprocessing steps included:

• Tokenization using the DistilBERT tokenizer
• Padding and truncation to a fixed maximum length
• Label encoding using Hugging Face ClassLabel

---

Training procedure

Hyperparameters

• Base model: distilbert-base-uncased
• Learning rate: 2e-5
• Train batch size: 32
• Eval batch size: 32
• Epochs: 10
• Optimizer: AdamW (betas = 0.9, 0.999, epsilon = 1e-8)
• Learning rate scheduler: Linear
• Loss function: Cross-Entropy with class weights
• Seed: 42

---

Training results

The best model was selected based on macro F1 score.

---

Framework versions

• Transformers: 4.44.2
• PyTorch: 2.6.0+cu124
• Datasets: 4.4.1
• Tokenizers: 0.19.1

---

Source code

Training and evaluation code is available on GitHub:
👉 https://github.com/Abdelrahmanemam01/Sentiment-Analysis