Downloads · 30 days
8
0% of all-time downloads
mt5-base-lm-adapt
DKYoon/mt5-base-lm-adapt
mt5-base-lm-adapt is a machine learning model from DKYoon. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
🤗 Language model initialized from mT5 and trained for an additional 100K steps on the Prefix LM objective using mC4 data.
All-time downloads
4K
Public
Parameters
582M
4.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin2.3 GB · 50%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- mt5
- Access
- Public
- Created
- Apr 13, 2023
- Updated
- Sep 5, 2023
- SHA
- 78d37314
Model card
From the Hugging Face model README
🤗 Language model initialized from mT5 and trained for an additional 100K steps on the Prefix LM objective using mC4 data.
Paper: Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation
Authors: Tu Vu, Aditya Barua, Brian Lester, Daniel Cer, Mohit Iyyer, Noah Constant
PyTorch port of the original Flax checkpoint at Google/T5X repository.
- Library
- transformers
- Type
- mt5
- License
- apache-2.0
- Created
- Apr 13, 2023
- Updated
- Sep 5, 2023