makini-hate-v1

MAKINI-Hate v1 — African Language Hate Speech Detection

Developed by Algedi Intelligence Labs (Nairobi, Kenya).

Model Summary

MAKINI-Hate v1 is a fine-tuned XLM-RoBERTa model for detecting hate speech in Swahili and French text. It is the first hate speech detection model trained on both Swahili (AfriHate) and French data with explicit documentation of African context gaps.

Binary classification: Hate vs Normal.

---

Labels

---

Performance (Test Set — 4,976 examples)

Overall

Benchmark — MAKINI-Hate v1 vs Existing Models

Evaluated on AfriHate Swahili test set (3,168 examples).

*AfroXLMR-76L results from Muhammad et al. (2025) AfriHate paper. Note: Comparisons are not perfectly controlled — AfriHate baselines use 3-class labels while MAKINI uses binary. Binary is an easier task. This limitation is documented transparently.

By Language

The 13-point gap between Swahili and French performance is a direct consequence of training data geography, not model architecture. This is a documented limitation and active research gap — see Known Limitations below.

---

Training Data

Label Mapping

AfriHate uses three labels (Hate, Abuse, Normal). The French superset is binary (1, 0). Both were mapped to binary for this model:

Important: The French superset binarized its source datasets, meaning what was originally Abusive was collapsed into 1 or 0 by the original authors. The model therefore has no French-language Abusive examples — the Abusive vs Hate distinction in French is an open research gap.

---

Training Approach

Architecture

Fine-tuned xlm-roberta-base (125M parameters) for binary sequence classification. XLM-RoBERTa was chosen for its multilingual pretraining across 100 languages including French and Swahili.

Why We Changed Approach Mid-Training

Attempt 1 (failed): Our first training run produced 0.881 F1 macro on the validation set across 4 epochs — numbers that looked strong. However, test set evaluation revealed the model was predicting only Normal for every input. Hate F1 = 0.00.

Root cause: The WeightedTrainer subclass was incompatible with the num_items_in_batch handling introduced in recent versions of the Transformers library. The weighted cross-entropy loss was being silently bypassed during training. The model learned to predict the majority class (Normal, ~60% of data) and achieved artificially high accuracy — a classic majority-class collapse on imbalanced data.

The fix: We rewrote compute_loss to explicitly move class weights to the same device as the logits (logits.device) and apply CrossEntropyLoss directly on reshaped tensors, bypassing the Trainer's internal loss handling entirely. This confirmed working weighted loss from epoch 1 (F1 0.858, both classes learning).

Lesson: Always evaluate per-class F1 on the test set immediately after training. Macro accuracy on validation alone is insufficient to detect majority-class collapse.

Hyperparameters

---

Usage

from transformers import pipeline

makini_hate = pipeline(
    "text-classification",
    model="Daudipdg/makini-hate-v1",
    return_all_scores=True
)

# Swahili
result = makini_hate("Mtu huyu anastahili kufa")
print(result)

# French
result = makini_hate("Ces gens ne méritent pas de vivre ici")
print(result)

---

Known Limitations

1. European French Bias (Critical)

The French training data comes entirely from European sources (France-based Twitter, NGO datasets). African French — as spoken in Senegal, Côte d'Ivoire, DRC, Cameroon, and other Francophone African countries — has distinct slang, code-switching patterns, and culturally-specific hate targets that this model has never seen.

Impact: The 13-point F1 gap between Swahili (0.92) and French (0.79) is directly attributable to this. A model trained on European French hate speech will systematically underperform on African French content.

What is needed: A purpose-built African French hate speech corpus annotated by native speakers from Francophone Africa. This does not currently exist. Building it is Algedi Intelligence Labs' highest-priority data collection goal for v2.

2. No Abusive/Hate Distinction in French

The French superset binarized all labels before release. The model cannot distinguish between Abusive and Hate in French — only in Swahili where AfriHate preserves this distinction. A future 3-class version requires properly annotated French data.

3. Code-Switching Not Covered

Swahili-English (Sheng), French-Wolof (as in AWOFRO), and Camfranglais are common in African online spaces. The model was not trained on code-switched text. We evaluated AWOFRO (3,510 Wolof-French tweets) as a potential training source but excluded it due to inconsistent annotation quality. This remains an open gap.

4. Tweet-Format Bias

Both training datasets are sourced from Twitter. Performance on other platforms (Facebook, WhatsApp forwards, forum posts) is untested and may degrade, particularly for longer texts.

5. Languages

Swahili and French only. No support for Hausa, Yoruba, Amharic, Zulu, or other African languages in v1.

---

What's Next: MAKINI-Hate v2 Roadmap

---

Explainability (Planned for v2)

MAKINI-Hate v2 will return structured output including:

{
  "label": "Hate",
  "confidence": 0.94,
  "cultural_markers": ["term_1", "term_2"],
  "explanation": "Detected hate targeting ethnicity",
  "recommendation": "High confidence — automated action defensible",
  "data_warning": null
}

This positions the model as a moderation assistant rather than a black-box classifier — critical for enterprise deployment where decisions must be explainable and defensible.

---

The African French Gap: A Research Note

This model quantifies for the first time the performance cost of using European French hate speech data to moderate African French content. The 13-point F1 gap (0.92 Swahili vs 0.79 French) is not a modeling failure — it is a data infrastructure failure.

No publicly available African French hate speech dataset exists as of March 2026. The French Hate Speech Superset authors themselves noted that their dataset overrepresents France relative to Francophone Africa. AWOFRO (Ndao et al., 2024) covers Wolof-French code-switching but not standard African French, and was excluded from this work due to annotation inconsistency.

This gap affects every NLP system deployed for content moderation in Francophone Africa. We document it here explicitly so that future work can measure against this baseline.

---

Citation

@model{wachira2026makinihate,
  title     = {MAKINI-Hate v1: African Language Hate Speech Detection},
  author    = {Wachira, David Maina},
  year      = {2026},
  publisher = {Algedi Intelligence Labs},
  url       = {https://huggingface.co/Daudipdg/makini-hate-v1},
  note      = {Fine-tuned on AfriHate (Swahili) and French Hate Speech 
               Superset. First hate speech model with explicit African 
               French context gap documentation.}
}

---

License

CC BY-NC 4.0 — Free for research and non-commercial use.
Commercial use requires explicit permission from Algedi Intelligence Labs.

---

Contact

David Maina Wachira — [email protected]
Algedi Intelligence Labs — makini.tech