SASCv2

Multilingual Hate Speech Detection — GloVe + BiLSTM (v2)

Task: Binary text classification (Hate / Non-Hate) Languages: English, Hindi, Hinglish (Hindi-English code-mixed) Architecture: Bidirectional LSTM with frozen GloVe embeddings Strategy: Hinglish → Hindi → English → Full (50 epochs per phase, 200 total)

---

Table of Contents

1. What This Experiment Does
2. The Dataset
3. Model Architecture
4. Training Strategy
5. Phase 1 — Hinglish
6. Phase 2 — Hindi
7. Phase 3 — English
8. Phase 4 — Full Dataset
9. Full Results Table
10. How to Use

---

1. What This Experiment Does

This is v2 of the SASC sequential transfer learning experiment.

v1 ran all 6 permutations of [English, Hindi, Hinglish] with 8 epochs per phase. v2 focuses on a single strategy — Hinglish → Hindi → English → Full — but trains for 50 epochs per phase (200 total). The key new addition: after every phase the model is evaluated on all three individual language test sets AND the full test set, giving a complete 4×4 cross-evaluation matrix showing how knowledge transfers across languages.

---

2. The Dataset

Dataset: tuklu/nprism

The dataset is dominated by English (50.8%). GloVe embeddings are also English-centric, which directly explains why the English phase produces the sharpest accuracy jump regardless of training order.

---

3. Model Architecture

Input: Text sequence (max 100 tokens)
       ↓
GloVe Embedding Layer (vocab: 50,000 × 300d) — FROZEN
       ↓
Bidirectional LSTM (128 units)
   → reads sentence left-to-right AND right-to-left
       ↓
Dropout (0.5)
       ↓
Dense Layer (64 neurons, ReLU)
       ↓
Output Layer (1 neuron, Sigmoid)
   → > 0.5 = Hate Speech  |  ≤ 0.5 = Non-Hate

• Optimizer: Adam
• Loss: Binary Cross-Entropy
• Max sequence length: 100 tokens
• Vocab size: 50,000

---

4. Training Strategy

The same model carries its weights through all 4 phases — no resets between languages. After each phase the model is evaluated against all three language-specific test sets and the full test set.

---

5. Phase 1 — Hinglish

Training on Hinglish only (2,908 samples, 50 epochs). The model starts cold. Hinglish is code-mixed and GloVe has limited coverage — the model learns from sequential patterns rather than word semantics.

Evaluation after Phase 1

The Hindi result (Recall=1.0, Specificity=0.0) shows the model predicts everything as hate on Hindi — it has no Hindi-specific knowledge yet. English performance is near-random. Hinglish F1=0.539 shows the model has learned something useful from its own language.

---

6. Phase 2 — Hindi

Training on Hindi (5,940 samples, 50 epochs). GloVe has limited Hindi coverage so the model must rely on contextual patterns. The struggle here is deliberate — it builds language-agnostic hate detection features.

Evaluation after Phase 2

Hindi F1 improves to 0.504. Hinglish drops — the model has partially overwritten Hinglish-specific patterns. English recall spikes (high false positives) showing the model is now biased toward predicting hate. This is the expected "catastrophic interference" that the Full phase resolves.

---

7. Phase 3 — English

Training on English (8,856 samples, 50 epochs). This is the turning point. GloVe embeddings align well with English — the model jumps sharply and the English-phase knowledge partially generalises back to the other languages.

Evaluation after Phase 3

English F1 leaps to 0.780 — the model now performs strongly on its native language. Full AUC reaches 0.691. Hinglish specificity collapses again (high recall, low precision) — the model over-predicts hate on unseen languages after English fine-tuning.

---

8. Phase 4 — Full Dataset

Training on the full shuffled dataset (17,704 samples, 50 epochs). This consolidation phase exposes the model to all three languages simultaneously, balancing out the per-language biases accumulated during sequential training.

Evaluation after Phase 4 (Final Model)

The Full phase restores balance across all languages. Hinglish specificity recovers to 0.721 (from 0.088 after English phase). Full-dataset AUC reaches 0.756 — the best of all phases. English performance is preserved at F1=0.771 while Hinglish and Hindi both improve substantially from their post-English-phase collapse.

---

9. Full Results Table

Complete 16-row cross-evaluation (Phase × Eval Language):

Key Observations

• English phase is the sharpest turning point — English F1 jumps from 0.596 (after Hindi) to 0.780 in one phase, driven by GloVe's English-centric embeddings.
• Starting from Hinglish forces generalisation from noise — the model reaches Hinglish F1=0.539 after only its own phase, a stronger start than Hinglish gets in most v1 orderings.
• Catastrophic interference is visible — Hinglish specificity drops from 0.791 → 0.747 → 0.088 as the model progressively shifts language bias. The Full phase restores it to 0.721.
• Final Full phase AUC = 0.756 matches the best v1 strategies despite a harder starting language, confirming the robustness of the Hinglish-first approach with deeper training.
• Hindi remains the hardest (F1=0.512 at final) — consistent with GloVe's limited Hindi vocabulary coverage.

---

10. How to Use

import json
import numpy as np
import tensorflow as tf
from tensorflow.keras.preprocessing.text import tokenizer_from_json
from tensorflow.keras.preprocessing.sequence import pad_sequences
from huggingface_hub import hf_hub_download

# Load tokenizer (from v1 repo — same dataset/split)
tokenizer_path = hf_hub_download(repo_id="tuklu/SASC", filename="tokenizer.json")
with open(tokenizer_path) as f:
    tokenizer = tokenizer_from_json(f.read())

# Load model
model_path = hf_hub_download(repo_id="tuklu/SASCv2", filename="model.h5")
model = tf.keras.models.load_model(model_path)

# Predict
texts = ["I hate all of them", "Have a great day!"]
sequences = tokenizer.texts_to_sequences(texts)
padded = pad_sequences(sequences, maxlen=100)
probs = model.predict(padded).flatten()

for text, prob in zip(texts, probs):
    label = "Hate Speech" if prob > 0.5 else "Non-Hate"
    print(f"{label} ({prob:.3f}): {text}")

---

Explainability — SHAP Analysis

We applied SHAP (SHapley Additive exPlanations) to the final trained model to understand which words drive hate speech predictions. A GradientExplainer runs on the BiLSTM sub-model (embedding layer bypassed — embeddings pre-computed as floats), with 200 background training samples, evaluated on all 4 test sets.

Full methodology, all strategy comparisons, and detailed word tables: SHAP_REPORT.md

Top SHAP Words — Final Model

Key Takeaways

• Hindi SHAP values are 10× smaller than English/Hinglish — GloVe has near-zero Hindi coverage; model relies on positional patterns, not word semantics
• Accusatory framing dominates full-dataset hate markers (blamed, criticized, advocating) — the 50-epoch Full phase learns that hate speech in this corpus often targets victims through blame/accusation rather than direct slurs
• "online" is the most consistent non-hate signal — informational/conversational context across all three languages
• Hinglish markers are semantically coherent (arey = hey/exclamation in abusive context, punish, interior) despite code-mixing — v2's 50 epochs on Hinglish-first produced stronger Hinglish feature learning than v1
• Spurious correlations remain (syntax, sine) — inherent limitation of non-contextual GloVe; a BERT-based model would resolve these

---

Related

• v1 (all 6 strategies, 8 epochs each): tuklu/SASC
• Dataset: tuklu/nprism

---

Citation

@misc{sasc2026,
  title={Multilingual Hate Speech Detection via Sequential Transfer Learning (v2)},
  author={tuklu},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/tuklu/SASCv2}
}