m2m100_dialect_ibo_en_mt

Model Description

This is a machine translation model from dialectal Igbo to English with the ability to handle regional language variations.

Paper: https://huggingface.co/papers/2405.00997

How to use

from transformers import AutoModelForSeq2SeqLM
from transformers import AutoTokenizer


model = AutoModelForSeq2SeqLM.from_pretrained('Ifyokoh/m2m100_dialect_ibo_en_mt')
tokenizer = AutoTokenizer.from_pretrained('Ifyokoh/m2m100_dialect_ibo_en_mt')

Citation

@inproceedings{emezue-etal-2024-igboapi,
    title = "The {I}gbo{API} Dataset: Empowering {I}gbo Language Technologies through Multi-dialectal Enrichment",
    author = "Emezue, Chris Chinenye  and
      Okoh, Ifeoma  and
      Mbonu, Chinedu Emmanuel  and
      Chukwuneke, Chiamaka  and
      Lal, Daisy Monika  and
      Ezeani, Ignatius  and
      Rayson, Paul  and
      Onwuzulike, Ijemma  and
      Okeke, Chukwuma Onyebuchi  and
      Nweya, Gerald Okey  and
      Ogbonna, Bright Ikechukwu  and
      Oraegbunam, Chukwuebuka Uchenna  and
      Awo-Ndubuisi, Esther Chidinma  and
      Osuagwu, Akudo Amarachukwu",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.1384",
    pages = "15932--15941",
    abstract = "The Igbo language is facing a risk of becoming endangered, as indicated by a 2025 UNESCO study. This highlights the need to develop language technologies for Igbo to foster communication, learning and preservation. To create robust, impactful, and widely adopted language technologies for Igbo, it is essential to incorporate the multi-dialectal nature of the language. The primary obstacle in achieving dialectal-aware language technologies is the lack of comprehensive dialectal datasets. In response, we present the IgboAPI dataset, a multi-dialectal Igbo-English dictionary dataset, developed with the aim of enhancing the representation of Igbo dialects. Furthermore, we illustrate the practicality of the IgboAPI dataset through two distinct studies: one focusing on Igbo semantic lexicon and the other on machine translation. In the semantic lexicon project, we successfully establish an initial Igbo semantic lexicon for the Igbo semantic tagger, while in the machine translation study, we demonstrate that by finetuning existing machine translation systems using the IgboAPI dataset, we significantly improve their ability to handle dialectal variations in sentences.",
}