Downloads · 30 days
194
21% of all-time downloads
AraBertMo_base_V3
Ebtihal/AraBertMo_base_V3
AraBertMo_base_V3 is a fill-mask model from Ebtihal. Use it when you need the model to fill a missing word. It is set up for transformers.
AraBERTMo is an Arabic pre-trained language model based on Google's BERT architechture. AraBERTMobase uses the same BERT-Base config. AraBERTMobase now comes in 10 new variants All models are available on the HuggingF…
All-time downloads
930
Public
Repo size
1.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin552 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- bert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Mar 15, 2022
- SHA
- f4097315
Try a prompt
Datasets
Model card
From the Hugging Face model README
Arabic BERT Model
AraBERTMo is an Arabic pre-trained language model based on Google's BERT architechture.
AraBERTMo_base uses the same BERT-Base config.
AraBERTMo_base now comes in 10 new variants
All models are available on the HuggingFace model page under the Ebtihal name.
Checkpoints are available in PyTorch formats.
Pretraining Corpus
`AraBertMo_base_V3' model was pre-trained on ~3 million words:
- OSCAR - Arabic version "unshuffled_deduplicated_ar".
Training results
this model achieves the following results:
| Task | Num examples | Num Epochs | Batch Size | steps | Wall time | training loss |
|---|---|---|---|---|---|---|
| Fill-Mask | 30024 | 3 | 64 | 1410 | 3h 10m 31s | 8.0201 |
Load Pretrained Model
You can use this model by installing torch or tensorflow and Huggingface library transformers. And you can use it directly by initializing it like this:
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("Ebtihal/AraBertMo_base_V3")
model = AutoModelForMaskedLM.from_pretrained("Ebtihal/AraBertMo_base_V3")
This model was built for master's degree research in an organization:
- University of kufa.
- Faculty of Computer Science and Mathematics.
- Department of Computer Science
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- Languages
- ar
- Created
- Mar 2, 2022
- Updated
- Mar 15, 2022