arabic-hate-speech-marbert-lora

Model Card for Arabic Hate Speech Classifier (MARBERT + LoRA)

A binary text classifier that flags Arabic text as hate or not_hate, fine-tuned from MARBERT using LoRA (parameter-efficient fine-tuning).

Model Details

Model Description

• Developed by: Rahaf Alruwaithi
• Model type: Binary text classification (Transformer encoder + LoRA adapters)
• Language(s) (NLP): Arabic (ar)
• License: Same terms as the base model, UBC-NLP/MARBERT
• Finetuned from model: UBC-NLP/MARBERT

Uses

Direct Use

Classifying a single piece of Arabic text as containing hate/offensive speech or not — for research, prototyping, or as one signal in a moderation pipeline.

Downstream Use

Content moderation triage, customer-support message flagging, or classroom anti-bullying monitoring (tracking repeated flagged messages per user rather than reading every message manually).

Out-of-Scope Use

Not designed to identify the type, severity, or target of offensive content, and not suitable as the sole, fully automated basis for removing content or taking action against a user — see Limitations below.

Bias, Risks, and Limitations

• Binary output only: it does not distinguish harassment, threats, or insults from one another.
• On a held-out test set, about 19% of true not_hate examples were misclassified as hate (false positives), and about 16% of true hate examples were missed (false negatives).
• Trained on a specific mix of dialectal and general Arabic text; performance may vary on dialects, domains, or writing styles not represented in that data.

Recommendations

Use with a human-in-the-loop review step rather than as an automated final decision-maker, especially for moderation actions that affect real users.

How to Get Started with the Model

from peft import PeftModel
from transformers import AutoModelForSequenceClassification, AutoTokenizer

base_model = AutoModelForSequenceClassification.from_pretrained("UBC-NLP/MARBERT", num_labels=2)
model = PeftModel.from_pretrained(base_model, "rahaf088/arabic-hate-speech-marbert-lora")
tokenizer = AutoTokenizer.from_pretrained("UBC-NLP/MARBERT")

text = "مثال على نص عربي"
inputs = tokenizer(text, truncation=True, padding="max_length", max_length=40, return_tensors="pt")
logits = model(**inputs).logits
prediction = logits.argmax(-1).item()  # 0 = not_hate, 1 = hate

Training Details

Training Data

Three Arabic sources merged, de-duplicated, and relabeled into a binary schema (hate / not_hate): L-HSAB (Levantine hate speech and abusive language dataset), a general Arabic hate/offensive-speech training set, and an Arabic Twitter dataset. Text was cleaned (diacritics and elongation removed, non-Arabic characters and URLs stripped, alef variants normalized) and split 80/10/10 into train/validation/test, stratified jointly by source and label.

Training Procedure

Preprocessing

Diacritic removal, tatweel (elongation) removal, URL and non-Arabic character stripping, alef normalization, whitespace cleanup.

Training Hyperparameters

• Training regime: fp16 mixed precision (GPU)
• Learning rate: 2e-5, batch size: 16, epochs: 4, weight decay: 0.01, warmup ratio: 0.1
• LoRA: r=8, alpha=16, dropout=0.1
• Loss: class-weighted cross-entropy (sklearn balanced class weights)
• Max sequence length: 40 tokens

Speeds, Sizes, Times

296,450 trainable parameters out of 163,139,332 (0.18% of the base model). Full training run: ~4 minutes on a single Colab GPU.

Evaluation

Testing Data, Factors & Metrics

Testing Data

A held-out 10% stratified split of the combined dataset, not seen during training or checkpoint selection.

Metrics

Accuracy, precision, recall, and macro-F1, plus F1 per class (hate, not_hate).

Results

Summary

Test-set results are within ~1 point of validation-set results across all metrics, indicating the model generalizes rather than overfitting to the training data.

Environmental Impact

• Hardware Type: Single GPU, Google Colab (T4/A100, shared cloud instance)
• Hours used: < 1 hour total across experiments
• Cloud Provider: Google Colab
• Carbon Emitted: Not formally calculated; expected to be minimal given the short training time and the parameter-efficient (LoRA) method used

Technical Specifications

Model Architecture and Objective

BERT-based encoder (MARBERT) with a linear binary classification head; class-weighted cross-entropy objective; LoRA low-rank adapters inserted into the attention layers instead of updating all base-model weights.

Compute Infrastructure

Hardware

Google Colab GPU

Software

transformers, peft, torch, scikit-learn, wandb

Model Card Authors

Rahaf Alruwaithi