yoruba-politeness-classifier

Yoruba Politeness Classifier

Project Overview

This project develops a text classification model for identifying politeness levels in Yoruba, an African language that remains comparatively underrepresented in NLP resources.

The model classifies Yoruba utterances into three categories:

• NEUTRAL
• POLITE
• IMPOLITE

Dataset

The project began with an original Yoruba conversational dataset and was subsequently adapted and prepared for model training.

The final working dataset contained 541 annotated Yoruba utterances.

Class distribution:

• NEUTRAL: 259
• POLITE: 240
• IMPOLITE: 39

The data were divided into:

• Training set: 430 examples
• Test set: 108 examples

Method

The dataset was prepared through data adaptation, cleaning, annotation, label encoding and train-test splitting.

The fine-tuned model is based on XLM-RoBERTa and was trained for 5 epochs.

Training Results

Final training loss: 0.136959

Final validation loss: 0.399099

Overall reported training loss: 0.268594

Benchmark Results

Accuracy: 89.81%

Weighted Precision: 0.89

Weighted Recall: 0.90

Weighted F1-score: 0.89

Macro F1-score: 0.74

Per-class results

Research Observation

The model performed strongly on NEUTRAL and POLITE utterances. Performance on IMPOLITE utterances was considerably lower, reflecting the smaller representation of this class in the dataset.

This result highlights the importance of balanced and linguistically diverse resources for Yoruba NLP.

Intended Use

The model is intended for research and educational purposes in Yoruba natural language processing, computational linguistics and African language technology.

Limitations

The dataset is relatively small and the IMPOLITE category is underrepresented. Therefore, the reported overall accuracy should be interpreted together with the macro-averaged metrics and per-class results.