Downloads · 30 days
72
0% of all-time downloads
bert-base-bg-cs-pl-ru-cased
DeepPavlov/bert-base-bg-cs-pl-ru-cased
bert-base-bg-cs-pl-ru-cased is a feature extraction model from DeepPavlov. Use it when you need embeddings to search or compare text. It is set up for transformers.
SlavicBERT\[1\] \(Slavic \(bg, cs, pl, ru\), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was bui…
All-time downloads
5.2M
Public
Repo size
3.6 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.bin714 MB · 50%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- Model type
- bert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Nov 8, 2021
- SHA
- 0ab00895
Model card
From the Hugging Face model README
bert-base-bg-cs-pl-ru-cased
SlavicBERT[1] (Slavic (bg, cs, pl, ru), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT.
08.11.2021: upload model with MLM and NSP heads
[1]: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. (2019). Tuning Multilingual Transformers for Language-Specific Named Entity Recognition. ACL anthology W19-3712.
- Task
- Feature Extraction
- Library
- transformers
- Type
- bert
- Languages
- bg, cs, pl, ru
- Created
- Mar 2, 2022
- Updated
- Nov 8, 2021