makini-bias-v2

MAKINI v2 — African Language Bias Detection Model

Model Description

MAKINI v2 is a 4-class bias detection model covering Swahili, French, isiZulu, and Hausa, fine-tuned on the AfricaBias dataset (60,234 sentences). It succeeds MAKINI v1 (Swahili/French only) — this is a new backbone and two additional languages, not an incremental v1.1.

Validated for production use: Swahili, French, isiZulu — all four classes. Hausa: neutral-vs-not detection only. Hausa's stereotype, counter-stereotype, and derogation classes are not reliable in this release — see Known Limitations before using Hausa output for anything beyond flagging "neutral or not."

Base Model

Davlan/afro-xlmr-base, not vanilla xlm-roberta-base (used in v1). Chosen because standard XLM-R's pretraining did not meaningfully cover isiZulu; AfroXLMR extends coverage to isiZulu, Hausa, and Chichewa (planned next language) in one backbone.

Training Approach

Two-stage fine-tune, same recipe as v1:

• Stage 1 — binary (neutral vs. biased), warm-start. F1 macro: 0.963
• Stage 2 — 4-class, initialized from Stage 1's encoder, weighted cross-entropy for class imbalance

Note: this release uses the same Swahili/French label distribution as v1 (24,220 neutral / 9,605 stereotype / 1,288 counter-stereotype / 172 derogation, combined). The separate counterstereotype/derogation rebalancing effort for Swahili/French has not yet been incorporated into any shipped model.

Labels

Performance — Held-Out Test Set (n=6,024)

Overall

The overall row is not a safe substitute for the per-language breakdown below — see Known Limitations.

Per Language

French (n=1,149) — accuracy 0.95, macro F1 0.96

Swahili (n=2,418) — accuracy 0.98, macro F1 0.74 (derogation support = 1, statistically meaningless — see Known Limitations)

isiZulu (n=1,486) — accuracy 0.99, macro F1 0.72 (counter-stereotype: no examples exist in source data — see Known Limitations)

Hausa (n=971) — accuracy 0.88, macro F1 0.32. Neutral detection works; the other three classes do not.

Known Limitations

Hausa: stereotype, counter-stereotype, and derogation detection are not usable. Recall on stereotype is 0.04 — the model misses 96% of actual Hausa stereotype content. Counter-stereotype recall is 0.00. Only neutral-vs-not-neutral is reliable for Hausa in this release. Root cause: Hausa's own 4-class split is heavily neutral-skewed (89%), and its 718/124/293 stereotype/counter-stereotype/derogation examples are a small fraction of the pooled multilingual training signal for those classes — global class weighting does not protect a single language's minority share within an already-weighted class. Addressing this needs either more Hausa non-neutral annotation or language-aware reweighting, not just longer training.

isiZulu has no counter-stereotype examples in the source data — this class cannot be evaluated for isiZulu and its true performance on isiZulu counter-stereotype content is unknown, not zero.

Swahili derogation test support is 1 example. The 0.00 F1 shown above reflects a single data point, not a real measurement of Swahili derogation performance.

isiZulu's domain field is unreliable for ~43% of source rows (see dataset card) — do not use domain-level breakdowns for isiZulu.

Limitations carried over from v1, not addressed by this release:

• Negation handling: stereotypes stated then negated are not reliably detected
• Sheng (informal Kenyan Swahili slang) is underrepresented in training data
• Implicit stereotyping conveyed through French sentence structure may be missed

Languages

Swahili, French, isiZulu, Hausa (Hausa scoped per Known Limitations above).

License

CC BY-NC 4.0. Commercial use requires explicit permission from Makini AI Limited.

Citation

@model{wachira2026makiniv2,
  title     = {MAKINI v2: African Language Bias Detection Model},
  author    = {Wachira, David Maina},
  year      = {2026},
  publisher = {Algedi Intelligence Labs},
  url       = {https://huggingface.co/MakiniAI/makini-v2},
}

Contact

David Maina Wachira — [email protected] — https://makini.tech