Downloads · 30 days
29
3% of all-time downloads
norwegian-t5-base
pere/norwegian-t5-base
norwegian-t5-base is a machine learning model from pere. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as cc-by-4.0.
This T5-base model is trained from scratch on a 19GB Balanced Bokmål-Nynorsk Corpus.
All-time downloads
1.1K
Public
Repo size
84 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.msgpack1.1 GB · 98%
At a glance
- Library
- transformers
- License
- cc-by-4.0
- Model type
- t5
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Sep 23, 2021
- SHA
- 685ed07e
Model card
From the Hugging Face model README
🇳🇴 Norwegian T5 Base model 🇳🇴
This T5-base model is trained from scratch on a 19GB Balanced Bokmål-Nynorsk Corpus.
Update: Due to disk space errors, the model had to be restarted July 20. It is currently still running.
Parameters used in training:
python3 ./run_t5_mlm_flax_streaming.py
--model_name_or_path="./norwegian-t5-base"
--output_dir="./norwegian-t5-base"
--config_name="./norwegian-t5-base"
--tokenizer_name="./norwegian-t5-base"
--dataset_name="pere/nb_nn_balanced_shuffled"
--max_seq_length="512"
--per_device_train_batch_size="32"
--per_device_eval_batch_size="32"
--learning_rate="0.005"
--weight_decay="0.001"
--warmup_steps="2000"
--overwrite_output_dir
--logging_steps="100"
--save_steps="500"
--eval_steps="500"
--push_to_hub
--preprocessing_num_workers 96
--adafactor
- Library
- transformers
- Type
- t5
- License
- cc-by-4.0
- Languages
- no
- Created
- Mar 2, 2022
- Updated
- Sep 23, 2021