arabic-ai-hybrid-xgboost

Arabic AI Text Detector — Hybrid (Character N-grams + Linguistic Features + XGBoost)

The best-performing model from the Arabic AI Text Detection project (CSC361, December 2025) — 93.16% internal accuracy, 86% accuracy on an external test set built from human text and AI outputs never seen during training.

This is not a standard transformers model — it's a scikit-learn/XGBoost pipeline:

1. A character-level TF-IDF vectorizer (2–5 grams, 20,000 features) captures punctuation habits, spacing, and morphological fragments.
2. 13 linguistic/stylometric features extracted with the Farasa Arabic NLP toolkit (POS ratios, average word length, type-token ratio, sentence length, unknown-token ratio — see the project report, Section 3.3.2).
3. Both feature blocks are concatenated (TF-IDF first, then the 13 linguistic features, in the order given in linguistic_feature_columns.pkl) and fed to an XGBClassifier.

Files:

• hybrid_xgb.pkl — the trained XGBClassifier (500 estimators)
• tfidf_vectorizer.pkl — fitted TfidfVectorizer (char, ngram_range=(2,5), max_features=20000)
• linguistic_feature_columns.pkl — ordered list of the 13 linguistic feature names

Labels: 0 = Human-written, 1 = AI-generated.

Try it interactively: 🤗 Space · Companion model: arabic-ai-arabert · Full report

Usage

Requires a Java runtime (Farasa is Java-based, wrapped by the farasapy Python package).

from huggingface_hub import hf_hub_download
import joblib, numpy as np

repo = "nawaf11570/arabic-ai-hybrid-xgboost"
model = joblib.load(hf_hub_download(repo, "hybrid_xgb.pkl"))
vectorizer = joblib.load(hf_hub_download(repo, "tfidf_vectorizer.pkl"))
feature_columns = joblib.load(hf_hub_download(repo, "linguistic_feature_columns.pkl"))

# see linguistic_features.py in the GitHub repo for extract_linguistic_features()
text = "هذا نص تجريبي لاختبار النموذج."
tfidf_vec = vectorizer.transform([text]).toarray()
ling_feats = extract_linguistic_features(text)  # dict of the 13 features
ling_vec = np.array([[ling_feats[c] for c in feature_columns]])

x = np.hstack([tfidf_vec, ling_vec])
proba = model.predict_proba(x)[0]
print("AI-generated" if proba.argmax() == 1 else "Human-written", proba.max())

Full runnable script: scripts/predict_hybrid.py in the GitHub repo.