Downloads · 30 days
12
19% of all-time downloads
ada-t5-small
SCIR-HI/ada-t5-small
ada-t5-small is a machine learning model from SCIR-HI. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
PseudoMD-1M dataset is the first artificially-real dataset for cross-modal molecule discovery, which consists of 1,020,139 pseudo molecule-description pairs. Every molecule is represented using its Canonical SMILES no…
All-time downloads
62
Public
Repo size
617 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin308 MB · 99%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- t5
- Access
- Public
- Created
- Dec 19, 2023
- Updated
- Dec 20, 2023
- SHA
- b762ccde
Model card
From the Hugging Face model README
Pre-trained T5-small model on PseudoMD-1M datasets.
PseudoMD-1M dataset is the first artificially-real dataset for cross-modal molecule discovery, which consists of 1,020,139 pseudo molecule-description pairs. Every molecule is represented using its Canonical SMILES notation, sourced from PubChem via the PUG View API. On average, each description within PseudoMD-1M contains 5.11 sentences, 106.47 words, and 165.07 tokens. We provide five examples in Appendix A in the paper.
Pre-training details
| Parameters | N |
|---|---|
| Corpus Size | 1,020,139 |
| Training Steps | 100,000 |
| Learning Rate | 1e-3 |
| Batch Size | 128 |
| Warm-up Steps | 1000 |
| Weight decay | 0.1 |
Example Usage
from transformers import AutoTokenizer, T5ForConditionalGeneration
model = T5ForConditionalGeneration.from_pretrained("SCIR-HI/ada-t5-small")
tokenizer = AutoTokenizer.from_pretrained("SCIR-HI/ada-t5-small", model_max_length=512)
Citation
@article{chen2023artificially,
title={From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery},
author={Chen, Yuhan and Xi, Nuwa and Du, Yanrui and Wang, Haochun and Jianyu, Chen and Zhao, Sendong and Qin, Bing},
journal={arXiv preprint arXiv:2309.05203},
year={2023}
}
- Library
- transformers
- Type
- t5
- License
- apache-2.0
- Languages
- en
- Created
- Dec 19, 2023
- Updated
- Dec 20, 2023