Downloads · 30 days
10
1% of all-time downloads
MorRoBERTa
otmangi/MorRoBERTa
MorRoBERTa is a fill-mask model from otmangi. Use it when you need the model to fill a missing word. It is set up for transformers.
MorRoBERTa is a Transformer-based Language Model designed specifically for the Moroccan Dialect. Developed by Moussaoui Otman and El Younoussi Yacine.
All-time downloads
1.5K
Public
Repo size
668 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin334 MB · 99%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- roberta
- Access
- Public
- Created
- Jun 12, 2023
- Updated
- Jun 12, 2023
- SHA
- ed755cb2
Try a prompt
Model card
From the Hugging Face model README
MorRoBERTa
MorRoBERTa is a Transformer-based Language Model designed specifically for the Moroccan Dialect. Developed by Moussaoui Otman and El Younoussi Yacine.
About MorRoBERTa
MorRoBERTa, designed specifically for the Moroccan dialect, is a scaled-down variant of the RoBERTa-base model. It comprises 6 layers, 12 attention heads, and 768 hidden dimensions. The training process spanned approximately 92 hours, covering 12 epochs on the complete training set. A vast corpus of six million Moroccan dialect sentences, amounting to 71 billion tokens, was used to train this model.
Usage
The model weights can be loaded using transformers library by HuggingFace.
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("otmangi/MorRoBERTa")
model = AutoModel.from_pretrained("otmangi/MorRoBERTa")
Acknowledgments
This research was supported through computational resources of HPC-MARWAN (www.marwan.ma/hpc) provided by the National Center for Scientific and Technical Research (CNRST). Rabat. Morocco.
Contact
For any inquiries, feedback, or requests, please feel free to reach out to :
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- Created
- Jun 12, 2023
- Updated
- Jun 12, 2023