Downloads · 30 days
8
21% of all-time downloads
jmvcoelho/t5-base-marco-lm-pretrain-2048
t5-base-marco-lm-pretrain-2048 is a machine learning model from jmvcoelho. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Model continually pre-trained on MS-MARCO (T5 span corruption task).
Downloads · 30 days
8
21% of all-time downloads
All-time downloads
39
Public
Repo size
893 MB
Likes
0
Public
Click a slice to open those files.
.bin892 MB · 100%
From the Hugging Face model README
Model continually pre-trained on MS-MARCO (T5 span corruption task).
Starting model: google-t5/t5-base
Training script: follows nanoT5 using the following huggingface class:
T5ForConditionalGenerationRoPE: https://github.com/cxcscmu/LongEmbeddingAnalysis/blob/main/OpenMatch/src/openmatch/modeling/rope_t5.py