Dialectal-Arabic-XLM-R-Base

Dialectal Arabic XLM-R Base

This is a repo of the language model used for "AdaSL: An Unsupervised Domain Adaptation framework for Arabic multi-dialectal Sequence Labeling". The state-of-the-art method for sequence labeling on multi-dialect Arabic.

About the Dialectal-Arabic-XLM-R-Base model

This model is an trained as a further pre-trained of XLM-RoBERTa base using the Masked-language modeling on a dialectal Arabic corpus.

About the Dialectal-Arabic-XLM-R-Base model training corpora

We have built a 5 million Tweets corpus from Twitter. The crawled tweets cover the dialects of the four Arabic world regions (EGY, GLF, LEV, and MAG regions), as well as MSA. The collected corpus consists of one million (1M) tweets per Arabic variant. We did not perform any text pre-processing on the tweets, except by removing tweets that have a small length (tweets containing less than four words).

Usage

The model weights can be loaded using transformers library by HuggingFace.

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("3ebdola/Dialectal-Arabic-XLM-R-Base")
model = AutoModel.from_pretrained("3ebdola/Dialectal-Arabic-XLM-R-Base")
text = "هذا مثال لنص باللغة العربية, يمكنك استعمال اللهجات العربية أيضا"
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)

Citation

@article{ELMEKKI2022102964,
title = {AdaSL: An Unsupervised Domain Adaptation framework for Arabic multi-dialectal Sequence Labeling},
journal = {Information Processing & Management},
volume = {59},
number = {4},
pages = {102964},
year = {2022},
issn = {0306-4573},
doi = {https://doi.org/10.1016/j.ipm.2022.102964},
url = {https://www.sciencedirect.com/science/article/pii/S0306457322000814},
author = {Abdellah {El Mekki} and Abdelkader {El Mahdaouy} and Ismail Berrada and Ahmed Khoumsi},
keywords = {Dialectal Arabic, Arabic natural language processing, Domain adaptation, Multi-dialectal sequence labeling, Named entity recognition, Part-of-speech tagging, Zero-shot transfer learning}
}