Downloads · 30 days
661
0% of all-time downloads
rut5-base
cointegrated/rut5-base
rut5-base is a machine learning model from cointegrated. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
This is a smaller version of the google/mt5-base model with only Russian and some English embeddings left.
All-time downloads
133K
Public
Parameters
244M
2.9 GB on disk
Likes
13
Public
Repo makeup
Click a slice to open those files.
.bin977 MB · 33%
At a glance
- Library
- transformers
- License
- mit
- Model type
- t5
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Mar 20, 2023
- SHA
- 49ce7aa5
Model card
From the Hugging Face model README
This is a smaller version of the google/mt5-base model with only Russian and some English embeddings left.
- The original model has 582M parameters, with 384M of them being input and output embeddings.
- After shrinking the
sentencepiecevocabulary from 250K to 30K (top 10K English and top 20K Russian tokens) the number of model parameters reduced to 244M parameters, and model size reduced from 2.2GB to 0.9GB - 42% of the original one.
The creation of this model is described in the post How to adapt a multilingual T5 model for a single language along with the source code.
- Library
- transformers
- Type
- t5
- License
- mit
- Languages
- ru, en, multilingual
- Created
- Mar 2, 2022
- Updated
- Mar 20, 2023