Downloads · 30 days
14
1% of all-time downloads
ru-e5-large
hivaze/ru-e5-large
ru-e5-large is a feature extraction model from hivaze. Use it when you need embeddings to search or compare text. It is set up for transformers.
This is a smaller version of the intfloat/multilingual-e5-large with only some Russian (Cyrillic in general) and English (fever) tokens (and embeddings) left.
All-time downloads
1.6K
Public
Parameters
375M
1.5 GB on disk
Likes
6
Public
Repo makeup
Click a slice to open those files.
.safetensors1.5 GB · 100%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- Model type
- xlm-roberta
- Access
- Public
- Created
- Feb 1, 2024
- Updated
- Feb 8, 2024
- SHA
- 549561fd
Model card
From the Hugging Face model README
About model creation
This is a smaller version of the intfloat/multilingual-e5-large with only some Russian (Cyrillic in general) and English (fever) tokens (and embeddings) left.
The model created in a similar way as described in this https://medium.com/m/global-identity-2?redirectUrl=https%3A%2F%2Ftowardsdatascience.com%2Fhow-to-adapt-a-multilingual-t5-model-for-a-single-language-b9f94f3d9c90 post.
The CulturaX dataset was used to search for the required tokens. As a result, out of 250k tokens of the original model, only 69,382 required were left.
Was the model trained in any way?
No. The tokenizer has been modified, and all changes to token identifiers have been corrected by moving embeddings in the model word_embeddings module to their new places, so the quality of this model on Cyrilic (and English) is exactly the same as the original one.
Why do we need this?
This allows you to use significantly less memory during training and also greatly reduces the weight of the model.
Authors
- Sergei Bratchikov (https://t.me/nlpwanderer)
- Task
- Feature Extraction
- Library
- transformers
- Type
- xlm-roberta
- Languages
- ru, uk, kk, be
- Created
- Feb 1, 2024
- Updated
- Feb 8, 2024