Downloads · 30 days
72
0% of all-time downloads
distilbert-base-es-multilingual-cased
Recognai/distilbert-base-es-multilingual-cased
distilbert-base-es-multilingual-cased is a fill-mask model from Recognai. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
This model is the Spanish extract of distilbert-base-multilingual-cased (https://huggingface.co/distilbert-base-multilingual-cased), a distilled version of the BERT base multilingual model. This model is cased: it doe…
All-time downloads
15.3K
Public
Parameters
63.8M
1 GB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.bin255 MB · 50%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- apache-2.0
- Model type
- distilbert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Jul 14, 2023
- SHA
- f7a3af27
Try a prompt
Datasets
Model card
From the Hugging Face model README
DistilBERT base multilingual model Spanish subset (cased)
This model is the Spanish extract of distilbert-base-multilingual-cased (https://huggingface.co/distilbert-base-multilingual-cased), a distilled version of the BERT base multilingual model. This model is cased: it does make a difference between english and English.
It uses the extraction method proposed by Geotrend described in https://github.com/Geotrend-research/smaller-transformers.
The resulting model has the same architecture as DistilmBERT: 6 layers, 768 dimension and 12 heads, with a total of 63M parameters (compared to 134M parameters for DistilmBERT).
The goal of this model is to reduce even further the size of the distilbert-base-multilingual multilingual model by selecting only most frequent tokens for Spanish, reducing the size of the embedding layer. For more details visit the paper from the Geotrend team: Load What You Need: Smaller Versions of Multilingual BERT.
- Task
- Fill-Mask
- Library
- transformers
- Type
- distilbert
- License
- apache-2.0
- Languages
- es
- Created
- Mar 2, 2022
- Updated
- Jul 14, 2023