XLM_Latin

XLM_Latin

Model Description

XLM_Latin is a transliteration-aware multilingual language model developed to investigate the impact of script unification on multilingual and cross-lingual hate speech detection. The model is based on the XLM-RoBERTa architecture and was further pretrained on multilingual transliterated corpora represented in a unified Latin script.

Intended Uses

The model is intended for:

• Multilingual hate speech detection
• Cross-lingual transfer learning
• Zero-shot learning
• Transliteration-based NLP research
• Multilingual representation learning

Languages

The model was trained on transliterated text from:

• Arabic
• Persian
• Hebrew
• Bengali
• Russian
• Chinese
• Korean
• Spanish

Training Data

The model was pretrained on 360,768 transliterated text samples collected from multiple sources, including news articles, sentiment datasets, social media content, and public corpora.

Model Architecture

• Base model: XLM-RoBERTa-base
• Model type: Masked Language Model (MLM)
• Tokenizer: SentencePiece (Unigram)
• Vocabulary size: 32,000

Evaluation

The model was evaluated using:

• Monolingual learning
• Cross-lingual transfer learning
• Zero-shot transfer learning
• Multilingual learning

Limitations

• Performance depends on transliteration quality.
• The model may inherit biases from training data.
• Intended primarily for research purposes.

Ethical Considerations

The model processes hate speech data that may contain offensive content. It should not be used as the sole decision-making system in high-risk applications.

How to Use the Model

Installation

Install the required libraries:

pip install transformers torch sentencepiece

Load the Model and Tokenizer

The pretrained XLM_Latin model can be loaded directly from the Hugging Face Hub:

from transformers import AutoTokenizer, AutoModelForMaskedLM

model_name = "GhadeerALbadani/XLM_Latin"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name)

Replace YOUR_USERNAME with your Hugging Face username or organization name.

---

Example: Tokenization

The tokenizer can be used to convert transliterated multilingual text into model inputs:

text = "ana la uhibbu hadha almujtama"

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    padding=True
)

print(inputs)

---

Example: Extracting Contextual Representations

The model can be used to generate contextual embeddings for transliterated multilingual text:

from transformers import AutoTokenizer, AutoModel

model_name = "GhadeerALbadani/XLM_Latin"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

text = "ana la uhibbu hadha almujtama"

inputs = tokenizer(
    text,
    return_tensors="pt"
)

outputs = model(**inputs)

embeddings = outputs.last_hidden_state

print(embeddings.shape)

---

Example: Masked Language Prediction

Since XLM_Latin is pretrained using the Masked Language Modeling (MLM) objective, it can predict masked tokens:

from transformers import pipeline

fill_mask = pipeline(
    "fill-mask",
    model="GhadeerALbadani/XLM_Latin",
    tokenizer="GhadeerALbadani/XLM_Latin"
)

result = fill_mask(
    "ana <mask> hadha almujtama"
)

print(result)

---

Fine-Tuning for Hate Speech Detection

XLM_Latin is intended to serve as a pretrained backbone model and can be further fine-tuned for multilingual hate speech detection tasks:

from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained(
    "YOUR_USERNAME/XLM_Latin",
    num_labels=2
)

The resulting fine-tuned model can then be used for monolingual, multilingual, cross-lingual, and zero-shot hate speech detection experiments.

Citation

@mastersthesis{XLM_Latin_2026,
  title={A Multilingual Model for Detecting Hate Speech on Unknown Language Script},
  author={Author Name},
  year={2026},
  school={University Name}
}