DarijaToxicityDetector

DarijaToxicityDetector — Moroccan Darija Toxicity Detection (Binary)

DarijaToxicityDetector is a BERT-based binary text classifier that detects toxic / offensive content in Moroccan Darija (Moroccan Arabic dialect, written in Arabic script). It is fine-tuned from SI2M-Lab/DarijaBERT on the OMCD_Typica.ai_Mix dataset, a curated blend of the public OMCD dataset and Typica.ai's proprietary culturally grounded annotations.

The model is released by Typica.ai as part of its applied research on culturally localized AI for underserved languages, and is open-sourced for educational and research purposes.

📄 Companion paper: A Comparative Benchmark of a Moroccan Darija Toxicity Detection Model (Typica.ai) and Major LLM-Based Moderation APIs (OpenAI, Mistral, Anthropic) — the benchmark shows that this culturally adapted model outperforms general-purpose LLM moderation APIs on Moroccan Darija toxicity detection.

Model Details

Labels

Intended Uses

Direct intended uses:

• Research on toxicity detection and content moderation for low-resource languages and Arabic dialects.
• Education: teaching NLP fine-tuning, evaluation, and culturally adapted model design.
• Benchmarking against general-purpose moderation systems (see companion paper).
• Prototyping moderation pipelines for Moroccan Darija user-generated content (comments, social media, forums).

Out-of-scope uses:

• ❌ Commercial deployment without a separate agreement with Typica.ai (license is non-commercial).
• ❌ Fully automated moderation decisions without human review — the model produces errors, especially on sarcasm.
• ❌ Text dominated by Latin script (Arabizi, French, English): such content was filtered out of training data.
• ❌ Other Arabic dialects or MSA — performance is not guaranteed outside Moroccan Darija.

How to Use

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="TypicaAI/DarijaToxicityDetector",
)

texts = [
    "هاد الفيديو ما عجبنيش بزاف", # expected: clean
    "مول هاد الفيديو باسل وما مربّيش", # expected: offensive
    "هاد الإنفلونسر مكلّخ غير كيخربق" # expected: offensive
]
print(classifier(texts, truncation=True, max_length=512))
# [{'label': 'clean', 'score': ...}, {'label': 'offensive', 'score': ...}]

Training Data: OMCD_Typica.ai_Mix

The model was trained on OMCD_Typica.ai_Mix (12,758 Moroccan Darija comments), built as follows:

• Backbone: OMCD — Offensive Moroccan Comments Dataset (Essefar et al., 2023), a widely cited, well-annotated public resource for Darija toxicity research.
• Enrichment: Typica.ai proprietary annotations covering culturally embedded toxicity — indirect insults, sarcasm, euphemisms, and culturally specific aggression that general-purpose models frequently miss.
• Preprocessing: sentences with >50% Latin-script characters were removed (to exclude French/English/Arabizi-dominated code-switching); Latin characters, symbols, and punctuation were cleaned while preserving linguistic meaning.
• Balancing: random undersampling of the majority (clean) class to reach a 1:1 clean/offensive ratio, preventing majority-class bias.

Splits

Each example carries sentence, label (ClassLabel: clean/offensive), idx, and origin (source provenance) fields.

The test split is publicly available for reproducibility in the benchmark GitHub repository. The proprietary training annotations are not released.

Training Procedure

• Base checkpoint: SI2M-Lab/DarijaBERT
• Objective: binary sequence classification (cross-entropy)
• Tokenization: DarijaBERT tokenizer, truncation at max_length=512, dynamic padding (DataCollatorWithPadding)

Hyperparameters:

Evaluation

Held-out test set (638 examples, balanced)

Benchmark vs. commercial moderation APIs

Original benchmark — from the companion paper (May 2025), on the OMCD_Typica.ai_Mix test split (n = 630):

Updated re-run — July 2026, same gold test set (n = 630, balanced), same inputs to all APIs, using each provider's then-current moderation endpoint (weighted precision / recall / F1):

Fourteen months after the original benchmark, the performance gap persists even against newer commercial models: the culturally adapted classifier still leads by ~18+ F1 points. General-purpose APIs continue to miss culturally nuanced toxicity (indirect insults, sarcasm, cultural idioms), while the specialized model maintains the best balance between catching toxic content and avoiding false positives.

Limitations & Bias

• Cross-dialectal noise: source data may include some non-Moroccan Arabic dialect examples.
• Annotation subjectivity: toxicity is culturally shaped; annotator judgment introduces some variance.
• Sarcasm false positives: highly sarcastic but benign messages can be flagged as offensive.
• Script coverage: Arabic script only; Arabizi/Latin-script Darija is out of scope.
• Temporal drift: online toxic language evolves; periodic re-training is recommended.

Ethical Considerations

This model deals with offensive content by design. It should support — not replace — human moderation. Misclassification can silence legitimate speech (false positives) or expose users to harm (false negatives). Deployers should implement human-in-the-loop review, appeal mechanisms, and threshold calibration appropriate to their community norms.

Citation

If you use this model, please cite:

@article{assoudi2025comparative,
  title   = {A Comparative Benchmark of a Moroccan Darija Toxicity Detection Model (Typica.ai) and Major LLM-Based Moderation APIs (OpenAI, Mistral, Anthropic)},
  author  = {Assoudi, Hicham},
  journal = {arXiv preprint arXiv:2505.04640},
  year    = {2025},
  url     = {https://arxiv.org/abs/2505.04640}
}

Contact

Hicham Assoudi — Founder & Applied AI Researcher, Typica.ai · PhD (AI/NLP) Typica.ai — Independent applied research initiative 📧 assoudi@typica.ai · Linkedin . 🌐 typica.ai · 🤗 TypicaAI on Hugging Face