Downloads · 30 days
40
2% of all-time downloads
bertin-base-gaussian-exp-512seqlen
bertin-project/bertin-base-gaussian-exp-512seqlen
bertin-base-gaussian-exp-512seqlen is a fill-mask model from bertin-project. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as cc-by-4.0.
This is a RoBERTa-base model trained from scratch in Spanish.
All-time downloads
2.6K
Public
Parameters
125M
15.3 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.msgpack4.7 GB · 82%
Parameter types
How the weights are stored.
F32125M · 100%
Try a prompt
Spaces
Model card
From the Hugging Face model README
This is a RoBERTa-base model trained from scratch in Spanish.
The training dataset is mc4 subsampling documents to a total of about 50 million examples. Sampling is biased towards average perplexity values (using a Gaussian function), discarding more often documents with very large values (poor quality) of very small values (short, repetitive texts).
This model takes the one using sequence length 128 and trains during 25.000 steps using sequence length 512.
Please see our main card for more information.
This is part of the Flax/Jax Community Week, organised by HuggingFace and TPU usage sponsored by Google.
Team members
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- License
- cc-by-4.0
- Languages
- es
- Created
- Mar 2, 2022
- Updated
- Feb 3, 2025