Downloads · 30 days
11
13% of all-time downloads
TamilXLM_Roberta
bytesizedllm/TamilXLM_Roberta
TamilXLM_Roberta is a machine learning model from bytesizedllm. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
We fine-tuned the base version of XLM-RoBERTa using Masked Language Modeling (MLM) to adapt it for handling transliteration and code-switching in Tamil-English dataset. The MLM task involves randomly masking a subset…
All-time downloads
88
Public
Parameters
278M
4.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.1 GB · 50%
At a glance
- Model type
- xlm-roberta
- Access
- Public
- Created
- Dec 16, 2024
- Updated
- Feb 4, 2025
- SHA
- 8d24ffc7
Model card
From the Hugging Face model README
We fine-tuned the base version of XLM-RoBERTa using Masked Language Modeling (MLM) to adapt it for handling transliteration and code-switching in Tamil-English dataset. The MLM task involves randomly masking a subset of input tokens and training the model to predict these masked tokens based on their context, allowing the model to learn enriched contextual embeddings tailored to the linguistic challenges of bilingual text.
To adapt XLM-RoBERTa effectively, the MLM training dataset was constructed from three key components:
- Original data: Contains monolingual text from Tamil AI4Bharath.
- Fully transliterated data: All words in the original data were transliterated into Roman script.
- Partially transliterated data: A randomly selected 20% to 70% of words in each sentence were transliterated into Roman script.
This model is a Tamil Masked Language Model (MLM) fine-tuned from the XLM-RoBERTa architecture.
Perplexity: 4.9
- Type
- xlm-roberta
- Created
- Dec 16, 2024
- Updated
- Feb 4, 2025