sentiment-sarcasm-detection-BanglaBERT

🌍 Bangla Sentiment & Sarcasm Dual-Head Model

Joint sentiment classification & sarcasm detection for imbalanced Bangla social media text

📁 Training Code & Scripts: GitHub

🤗 Model Weights & Inference: Hugging Face

📖 Paper: Zenodo

---

📦 Repository Contents

♻️ Reproducibility

• ✅ Fixed random seed (42) for all experiments
• ✅ 5-fold stratified cross-validation with bootstrap CIs
• ✅ All thresholds tuned on validation folds only (no test leakage)
• ✅ Code, data, and model weights publicly available

📖 Model Details

This model implements a calibrated multitask framework for joint sentiment and sarcasm detection in low-resource Bangla social media text. It addresses severe class imbalance and pragmatic ambiguity through:

• 🔹 Dual-head architecture: Shared BanglaBERT encoder → 256-dim projection layer → independent sentiment & sarcasm classification heads
• 🔹 Dynamic loss scheduling: Fold-adaptive inverse-frequency α scaling + linear γ decay (2.5 → 0.8) for epoch-aware hard-example mining
• 🔹 Post-hoc threshold calibration: Per-class decision boundaries optimized on validation folds to prevent majority-class bias
• 🔹 Data augmentation: BanglaT5 paraphrasing applied offline to enrich minority classes

The model was trained on 6,507 manually annotated cricket fan comments from Bangladesh’s 2023 ICC World Cup campaign, spanning Facebook and YouTube discourse.

---

🛠️ How to Use

Installation

pip install transformers torch numpy huggingface_hub

Inference Example (with calibrated thresholds)

import torch
import numpy as np
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer
from model_architecture import DualHeadModel

REPO_ID = "ahs95/sentiment-sarcasm-detection-BanglaBERT"

# Load tokenizer & model
tokenizer = AutoTokenizer.from_pretrained(REPO_ID)
model = DualHeadModel(num_sentiment_classes=4, num_sarcasm_classes=2)

model_path = hf_hub_download(repo_id=REPO_ID, filename="model.pth")
model.load_state_dict(torch.load(model_path, map_location="cpu", weights_only=True))
model.eval()

# Load calibrated thresholds
sent_thresholds = np.load(hf_hub_download(repo_id=REPO_ID, filename="sent_thresholds.npy"))
sarc_thresholds = np.load(hf_hub_download(repo_id=REPO_ID, filename="sarc_thresholds.npy"))

sentiment_labels = ["Positive", "Neutral", "Negative", "Mixed"]
sarcasm_labels = ["Sarcastic", "Non-Sarcastic"]  # Index 0 = Sarcastic

def predict(text, max_len=512):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=max_len, padding="max_length")
    
    with torch.no_grad():
        # DualHeadModel returns tuple: (sent_logits, sarc_logits)
        sent_logits, sarc_logits = model(inputs["input_ids"], inputs["attention_mask"])
    
    sent_probs = torch.softmax(sent_logits.squeeze(0), dim=-1)
    sarc_prob = torch.sigmoid(sarc_logits.squeeze(0))[0]  # P(Sarcastic)
    
    # Apply calibrated thresholds
    sent_pred = "Neutral"  # fallback
    for i, prob in enumerate(sent_probs):
        if prob >= sent_thresholds[i]:
            sent_pred = sentiment_labels[i]
            break
    
    sarc_pred = sarcasm_labels[0] if sarc_prob >= sarc_thresholds[0] else sarcasm_labels[1]
    
    return {
        "sentiment": sent_pred,
        "sarcasm": sarc_pred,
        "confidence": {
            "sentiment": sent_probs.tolist(),
            "sarcasm": [sarc_prob.item(), 1 - sarc_prob.item()]
        }
    }

# Test
result = predict("বাংলাদেশ জিতবে ২০৫০ বিশ্বকাপ, তখন আমি আর বেঁচে থাকব না।")
print(result)
# Expected: {'sentiment': 'Negative', 'sarcasm': 'Sarcastic', 'confidence': {...}}

📦 Note: The DualHeadModel class definition is available in the training repository. Copy model_architecture.py to your local environment before running the inference example.

---

📊 Evaluation Results

Evaluation followed a 5-fold stratified cross-validation protocol. Metrics are macro/weighted averaged across folds. Confidence intervals computed via 2,000 bootstrap resamples.

🎯 Sentiment Analysis (4-class)

😏 Sarcasm Detection (2-class)

📉 Ablation Baseline: Vanilla Cross-Entropy yields W-F1=0.69 (Sent) & 0.61 (Sarc) with complete minority-class collapse (Neutral/Mixed F1: 0.00).

---

🧪 Training Details

---

⚠️ Limitations & Bias

1. Domain Specificity: Trained exclusively on cricket fan discourse. Performance may degrade on political, e-commerce, or formal Bangla text without domain adaptation.
2. Pragmatic Reasoning: Struggles with lexical-pragmatic inversion, culturally embedded metaphors (e.g., মীরজাফর), and negation-driven intensification (লজ্জা নেই).
3. Representation Bottleneck: Relies on [CLS] pooling, which compresses dual-polarity utterances and obscures long-range pragmatic dependencies.
4. No Multimodal/Code-Mix Support: Emojis, memes, and Bangla-English code-switching are not explicitly modeled. Future work will integrate adapter-based multimodal extensions.
5. Threshold Calibration: Post-hoc procedure; not differentiable. Embedding cost-sensitive objectives directly into training may yield further gains.

🔍 Error Analysis: 50.1% of misclassifications are sarcasm-related, primarily due to hyperbolic non-sarcastic comments sharing pragmatic features with irony.

---

📚 Citation

If you use this model or dataset in your research, please cite:

@article{banglasentimentsarcasm,
  title={Sentiment and Sarcasm Detection in Bangla: A Calibrated Multitask Framework for Imbalanced Cricket Discourse},
  author={Arshadul Hoque and Nasrin Sultana and Risul Islam Rasel},
  year={2026},
  publisher={Zenodo},
  doi={10.5281/zenodo.20307593}
}

---

🤝 Contact

• 📧 [email protected]

---