Downloads · 30 days
13.2K
1% of all-time downloads
electra-small-nordic
jonfd/electra-small-nordic
electra-small-nordic is a machine learning model from jonfd. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as cc-by-4.0.
This model was pretrained on the following corpora: The Icelandic Gigaword Corpus (IGC) The Icelandic Common Crawl Corpus (IC3) The Icelandic Crawled Corpus (ICC) The Multilingual Colossal Clean Crawled Corpus (mC4) -…
All-time downloads
991K
Public
Repo size
264 MB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.h588.1 MB · 50%
At a glance
- Library
- transformers
- License
- cc-by-4.0
- Model type
- electra
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Jan 31, 2022
- SHA
- fb032b45
Model card
From the Hugging Face model README
Nordic ELECTRA-Small
This model was pretrained on the following corpora:
- The Icelandic Gigaword Corpus (IGC)
- The Icelandic Common Crawl Corpus (IC3)
- The Icelandic Crawled Corpus (ICC)
- The Multilingual Colossal Clean Crawled Corpus (mC4) - Icelandic, Norwegian, Swedish and Danish text obtained from .is, .no, .se and .dk domains, respectively
The total size of the corpus after document-level deduplication and filtering was 14.82B tokens, split equally between the four languages. The model was trained using a WordPiece tokenizer with a vocabulary size of 96,105 for one million steps with a batch size of 256, and otherwise with default settings.
Acknowledgments
This research was supported with Cloud TPUs from Google's TPU Research Cloud (TRC).
This project was funded by the Language Technology Programme for Icelandic 2019-2023. The programme, which is managed and coordinated by Almannarómur, is funded by the Icelandic Ministry of Education, Science and Culture.
- Library
- transformers
- Type
- electra
- License
- cc-by-4.0
- Languages
- is, no, sv, da
- Created
- Mar 2, 2022
- Updated
- Jan 31, 2022