nuha-ajp-trinary

nuha-ajp-trinary-onnx

Model Summary

nuha-ajp-trinary is an Arabic text classifier that categorises Jordanian social media comments into three classes based on the NUHA methodology for online gender-based violence (OGBV). It fine-tunes nuha-ajp-mlm — a domain-adapted Arabic BERT — and outputs one of:

This model was developed as part of a pilot proof-of-concept for the NUHA project by the Jordan Open Source Association (JOSA). It is the production model behind the NUHA analysis platform.

A lightweight, ONNX-optimised 4-layer classifier trained on the same task is available at thejosango/nuha-ajp-trinary-onnx.

For a simpler binary classifier (hate speech / non-hate speech), see nuha-ajp-binary.

Uses

Direct Use

Classifying Arabic social media comments for online gender-based violence, particularly for Jordanian Arabic content from Facebook and X (Twitter).

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="thejosango/nuha-ajp-trinary",
    tokenizer="thejosango/nuha-ajp-trinary",
)

result = classifier("اخرسي يا غبية")
print(result)
# [{'label': 'Gender Based Violence', 'score': ...}]

For batch inference:

comments = ["يعطيكم العافية", "أنتِ ساحرة", "اخرسي يا غبية"]
results = classifier(comments)
for comment, result in zip(comments, results):
    print(f"{result['label']} ({result['score']:.2f}): {comment}")

Using the ONNX Version

For faster CPU inference, use the ONNX export:

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

model = ORTModelForSequenceClassification.from_pretrained("thejosango/nuha-ajp-trinary-onnx")
tokenizer = AutoTokenizer.from_pretrained("thejosango/nuha-ajp-trinary-onnx")
classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)

Out-of-Scope Use

• Other Arabic dialects: The model was trained primarily on Jordanian Arabic. Performance on Egyptian, Gulf, or Modern Standard Arabic is not validated.
• Other hate speech targets: NUHA is calibrated for online gender-based violence. It is not designed to detect hate speech targeting race, religion, or other demographics.
• High-stakes automated decisions: Given the moderate performance (F1 ≈ 0.54) and pilot nature of this work, the model should not be used as the sole decision-maker in content moderation systems without human review.

Bias, Risks, and Limitations

• Pilot annotation quality: Training labels were produced in an exploratory annotation effort with variable inter-annotator agreement. The model inherits noise from that process, which is reflected in the moderate F1 score.
• Three-class difficulty: Distinguishing Offensive Language from Gender Based Violence is a genuinely difficult subtask. The Offensive Language class is small (≈2% of training data) and the model may struggle with it.
• Colloquial Arabic only: The aggressive text cleaning (Arabic-only filtering) means the model has never seen URLs, numbers, punctuation, or Latin-script text.
• Imbalanced classes: The training data is dominated by Not Online Violence (≈59%), with Offensive Language being very sparse (≈2%). Data augmentation was applied but class imbalance remains a factor.

Training Details

Training Data

Fine-tuned on the methodology configuration of thejosango/nuha-ajp-dataset, which applies the three-class NUHA categorisation scheme to the original annotations.

Preprocessing

At training and inference time, the following normalisation is applied to input text (in addition to the dataset-level Arabic-only filtering):

1. URLs replaced with [رابط] token
2. @mentions replaced with [مستخدم] token
3. Email addresses replaced with [بريد] token
4. Numbers removed
5. Punctuation removed
6. Arabic diacritics (harakat) removed
7. Whitespace normalised

Hyperparameters

Evaluation Results

Evaluated on the validation split of thejosango/nuha-ajp-dataset (methodology configuration):

The lower recall relative to precision suggests the model is conservative — it tends to under-predict Gender Based Violence rather than over-predict it. This reflects both the difficulty of the three-class task and the limited size of the pilot training corpus.

---

This model was developed as part of an initial pilot study. Performance metrics reflect the complexity of the task and the proof-of-concept nature of this system.