Downloads · 30 days
17
3% of all-time downloads
byt5-base-dv
monsoon-nlp/byt5-base-dv
byt5-base-dv is a machine learning model from monsoon-nlp. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Pretrained from scratch on Dhivei (language of the Maldives) with ByT5, Google's new byte-level tokenizer strategy.
All-time downloads
609
Public
Parameters
679M
8.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.h52.7 GB · 33%
At a glance
- Library
- transformers
- Model type
- t5
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Jan 21, 2025
- SHA
- 3058ac37
Model card
From the Hugging Face model README
byt5-base-dv
Pretrained from scratch on Dhivei (language of the Maldives) with ByT5, Google's new byte-level tokenizer strategy.
Use byt5-dv for now; this is less accurate
Corpus: Sofwath's Dhivehi corpus https://github.com/Sofwath/DhivehiDatasets
Pretraining Notebook: https://colab.research.google.com/drive/1ERIZ1PyHn-yN_jo7dTQeODn22vrt-d1d?usp=sharing
Fine-tuning Demo
On Dhivehi news classification task
https://colab.research.google.com/drive/11u5SafR4bKICmArgDl6KQ9vqfYtDpyWp?usp=sharing
Issues
There was an issue with the vocabulary size, final layer, and/or accuracy on fine-tuning.
- Library
- transformers
- Type
- t5
- Languages
- dv
- Created
- Mar 2, 2022
- Updated
- Jan 21, 2025