Downloads · 30 days
115
4% of all-time downloads
bertin-roberta-large-spanish
flax-community/bertin-roberta-large-spanish
bertin-roberta-large-spanish is a fill-mask model from flax-community. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as cc-by-4.0.
BERTIN is a series of BERT-based models for Spanish. This one is a RoBERTa-large model trained from scratch on the Spanish portion of mC4 using Flax, including training scripts.
All-time downloads
3.2K
Public
Parameters
125M
12.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin499 MB · 40%
Parameter types
How the weights are stored.
F32125M · 100%
Try a prompt
Model card
From the Hugging Face model README
NOTE: This repository is now superseded by https://huggingface.co/bertin-project/bertin-roberta-base-spanish. This model corresponds to the beta version of the model using stepwise over sampling trained for 200k steps with 128 sequence lengths. Version 1 is now available and should be used instead.
BERTIN
BERTIN is a series of BERT-based models for Spanish. This one is a RoBERTa-large model trained from scratch on the Spanish portion of mC4 using Flax, including training scripts.
This is part of the Flax/Jax Community Week, organised by HuggingFace and TPU usage sponsored by Google.
Spanish mC4
The Spanish portion of mC4 containes about 416 million records and 235 billion words.
$ zcat c4/multilingual/c4-es*.tfrecord*.json.gz | wc -l
416057992
$ zcat c4/multilingual/c4-es*.tfrecord-*.json.gz | jq -r '.text | split(" ") | length' | paste -s -d+ - | bc
235303687795
Team members
- Javier de la Rosa (versae)
- Eduardo González (edugp)
- Paulo Villegas (paulo)
- Pablo González de Prado (Pablogps)
- Manu Romero (mrm8488)
- María Grandury (mariagrandury)
Useful links
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- License
- cc-by-4.0
- Languages
- es
- Created
- Mar 2, 2022
- Updated
- Jul 26, 2023