raven-emotion-distilbert

Raven Emotion DistilBERT

A fine-tuned DistilBERT model for 6-class emotion classification, built for Raven AI — an emotionally aware AI assistant.

This model classifies text into 6 emotions: happy, sad, anxious, angry, confused, neutral.

Performance

Key result: This 67M parameter model outperforms a 70B parameter LLM by +4.33% on emotion classification, proving that task-specific fine-tuning beats general-purpose prompting.

Quick Start

from transformers import pipeline

classifier = pipeline("text-classification", model="Fynman-stack/raven-emotion-distilbert", top_k=None)

result = classifier("I'm so stressed about my exam tomorrow")
print(result)
# [[{'label': 'anxious', 'score': 0.95}, {'label': 'sad', 'score': 0.02}, ...]]

Or load the model directly:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Fynman-stack/raven-emotion-distilbert")
model = AutoModelForSequenceClassification.from_pretrained("Fynman-stack/raven-emotion-distilbert")

EMOTIONS = ["happy", "sad", "anxious", "angry", "confused", "neutral"]

def detect_emotion(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128, padding=True)
    with torch.no_grad():
        outputs = model(**inputs)
    return EMOTIONS[torch.argmax(outputs.logits, dim=1).item()]

print(detect_emotion("I just cleared my exam!"))  # happy
print(detect_emotion("I'm furious at this situation"))  # angry

Labels

Training Details

Phase 1: Initial Training on GoEmotions

• Base model: distilbert-base-uncased (67M parameters)
• Dataset: GoEmotions — Google's 28-emotion dataset, mapped to 6 categories
• Epochs: 3 | Batch size: 16 | Learning rate: 2e-5 | Optimizer: AdamW (weight decay 0.01)

Phase 2: Domain Adaptation on Custom Dataset

The model was further trained on ~12,343 samples of Indian English, Hinglish (Hindi-English), American English, and British English conversational text to adapt it for real-world student conversations.

• Learning rate: 5e-6 (reduced to prevent catastrophic forgetting)
• Early stopping: Patience of 2 epochs
• Warmup: 10% of total training steps
• Gradient clipping: 1.0

Domain adaptation impact: Accuracy jumped from 64.38% to 97.62% (+33.24%) on the target domain.

GoEmotions Label Mapping

The original 28 GoEmotions labels were mapped to 6 categories:

Use Cases

• Emotionally aware chatbots — Adjust response tone based on user emotion
• Mental health applications — Detect distress, anxiety, or anger in user messages
• Customer support — Route frustrated or confused customers to appropriate agents
• Social media monitoring — Track emotional sentiment across conversations
• Education platforms — Detect student frustration or confusion in real-time

About Raven AI

This model powers Raven AI, an emotionally aware AI assistant that adapts its tone, persona, and response style based on detected user emotion. Raven includes crisis detection, multi-chat management, image understanding, voice input, document processing, and 20+ other features.

• Try it live (HuggingFace Space): huggingface.co/spaces/Fynman-stack/raven-ai
• Streamlit Cloud: raven-ai-new.streamlit.app
• GitHub: github.com/Fynman-stack/raven-ai

Model Architecture

• Base: DistilBERT (6 layers, 12 attention heads, 768 hidden dim)
• Parameters: 67M
• Task head: Sequence classification (6 classes)
• Max sequence length: 128 tokens
• Format: Safetensors (FP32)

Limitations

• Trained primarily on English and Hinglish text — may not generalize well to other languages
• Emotion categories are coarse-grained (6 classes) — may miss nuanced emotional states
• Performance on formal/academic text may differ from conversational text
• Not a diagnostic tool — should not be used as a substitute for professional mental health assessment

Citation

@misc{raha2026raven,
  title={Raven AI: An Emotionally Aware AI Assistant with Fine-tuned DistilBERT},
  author={Soumyadip Raha},
  year={2026},
  url={https://huggingface.co/Fynman-stack/raven-emotion-distilbert}
}

License

MIT