Downloads · 30 days
18
0% of all-time downloads
mt5-small-lm-adapt
DKYoon/mt5-small-lm-adapt
mt5-small-lm-adapt is a machine learning model from DKYoon. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
🤗 Language model initialized from mT5 and trained for an additional 100K steps on the Prefix LM objective using mC4 data.
All-time downloads
3.9K
Public
Parameters
300M
2.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.2 GB · 50%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- mt5
- Access
- Public
- Created
- Apr 13, 2023
- Updated
- Sep 5, 2023
- SHA
- 09cdab37
Model card
From the Hugging Face model README
🤗 Language model initialized from mT5 and trained for an additional 100K steps on the Prefix LM objective using mC4 data.
Paper: Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation
Authors: Tu Vu, Aditya Barua, Brian Lester, Daniel Cer, Mohit Iyyer, Noah Constant
PyTorch port of the original Flax checkpoint at Google/T5X repository.
- Library
- transformers
- Type
- mt5
- License
- apache-2.0
- Created
- Apr 13, 2023
- Updated
- Sep 5, 2023