Downloads · 30 days
16
18% of all-time downloads
flaubert-base-kreyol-RH
Rziane/flaubert-base-kreyol-RH
flaubert-base-kreyol-RH is a machine learning model from Rziane. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gpl-3.0.
Continued MLM pre-training of flaubert/flaubertbasecased on a custom text corpus.
All-time downloads
87
Public
Parameters
138M
2.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt1.1 GB · 40%
At a glance
- License
- gpl-3.0
- Model type
- flaubert
- Access
- Public
- Created
- Apr 24, 2026
- Updated
- Apr 25, 2026
- SHA
- 2b87820d
Base models
Model card
From the Hugging Face model README
Rziane/flaubert-base-kreyol-RH
Continued MLM pre-training of flaubert/flaubert_base_cased on a custom text corpus.
Training
- base model:
flaubert/flaubert_base_cased - epochs: 8.0
- batch size (per device): 64
- gradient accumulation: 1
- learning rate: 5e-05
- scheduler: cosine (warmup 0.06)
- weight decay: 0.01
- max length: 128
- mlm probability: 0.15
- precision: fp16
- seed: 42
Final eval
- eval_loss: 1.2977
- eval_runtime: 295.1399
- eval_samples_per_second: 804.4630
- eval_steps_per_second: 6.2850
- epoch: 8.0000
- perplexity: 3.6607
Usage
from transformers import AutoTokenizer, AutoModelForMaskedLM
tok = AutoTokenizer.from_pretrained('Rziane/flaubert-base-kreyol-RH')
model = AutoModelForMaskedLM.from_pretrained('Rziane/flaubert-base-kreyol-RH')
- Type
- flaubert
- License
- gpl-3.0
- Languages
- ht
- Created
- Apr 24, 2026
- Updated
- Apr 25, 2026