fine_tuned_bert_emotions_large

fine_tuned_bert_emotions_large

Model summary

• Base: bert-large-uncased
• Task: multi-label emotion classification (GoEmotions-level emotions)
• Fine-tuning: tri-tower setup with contrastive context/label alignment
• Max length: 256
• Labels: same 28 GoEmotions emotions (excluding example_very_unclear)

Intended use

• Classify short texts (social posts, chats) with multiple emotions.
• Not for medical/mental-health diagnosis; avoid high-stakes use without human review.

Training data

• GoEmotions dataset
• Preprocessing: standard HF tokenizer, lowercased, truncation at 256 tokens.

Training procedure

• Optimizer: AdamW, LR 5e-5 (context head 2e-5), cosine scheduler, warmup 10%.
• Batch size: 8 (eval 32), epochs: 40 (early stop on val_f1_micro).
• Losses: BCE-with-logits for context, InfoNCE contrastive temperature 0.07, context loss weight 1.0.
• Regularization: dropout 0.1–0.2 (head), label smoothing 0.05.
• Hardware: NVIDIA GPU (NVIDIA GeForce RTX 5090 (sm_120)).

Evaluation

Replace with your best numbers:

• Test F1 (micro): 0.53
• Test F1 (macro): 0.41
• Precision (micro): 0.47
• Accuracy: 0.38
• Thresholding: per-label tuned on validation split.

How to use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "sdeakin/fine_tuned_bert_emotions_large"
tok = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "I’m excited but a bit nervous about tomorrow!"
enc = tok(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
    logits = model(**enc).logits
probs = torch.sigmoid(logits)[0]
label_map = model.config.id2label
preds = [(label_map[i], probs[i].item()) for i in range(len(probs))]
print(sorted(preds, key=lambda x: x[1], reverse=True)[:5])