Downloads · 30 days
15
2% of all-time downloads
bert-medium-luxembourgish
raduion/bert-medium-luxembourgish
bert-medium-luxembourgish is a fill-mask model from raduion. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.
Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.
All-time downloads
940
Public
Repo size
568 MB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.h5568 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- mit
- Model type
- bert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Jan 9, 2022
- SHA
- f8366494
Model card
From the Hugging Face model README
BERT Medium for Luxembourgish
Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.
The MLM objective was trained. The BERT model has parameters L=8 and H=512. Vocabulary has 70K word pieces.
Final loss scores, after 3 epochs:
- Final train loss: 4.230
- Final train perplexity: 68.726
- Final validation loss: 4.074
- Final validation perplexity: 58.765
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- License
- mit
- Languages
- lu
- Created
- Mar 2, 2022
- Updated
- Jan 9, 2022