Downloads · 30 days
3
9% of all-time downloads
ro-modernBERT-phase-d-wwm
alina0195/ro-modernBERT-phase-d-wwm
ro-modernBERT-phase-d-wwm is a machine learning model from alina0195. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Romanian ModernBERT-base produced by continual pretraining of ModernBERT-base on the Romanian FineWeb2 corpus with a custom 42k Romanian tokenizer.
All-time downloads
35
Public
Parameters
143M
1.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin574 MB · 50%
At a glance
- License
- apache-2.0
- Model type
- modernbert
- Access
- Public
- Created
- May 28, 2026
- Updated
- May 28, 2026
- SHA
- 75507627
Model card
From the Hugging Face model README
alina0195/ro-modernBERT-phase-d-wwm
Romanian ModernBERT-base produced by continual pretraining of ModernBERT-base on the Romanian FineWeb2 corpus with a custom 42k Romanian tokenizer.
This is the Phase D checkpoint (whole-word masking): context-extended to 8192 tokens in Phase C, then cooled down with a 1-sqrt LR schedule in Phase D.
- Backbone: ModernBERT-base (22 layers, hidden 768, 12 heads)
- Tokenizer: custom Romanian BPE, vocab=42240
- Max sequence length: 8192
- Training framework: MosaicML Composer
- Type
- modernbert
- License
- apache-2.0
- Languages
- ro
- Created
- May 28, 2026
- Updated
- May 28, 2026