Downloads · 30 days
9
1% of all-time downloads
scibart-large
uclanlp/scibart-large
scibart-large is a feature extraction model from uclanlp. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as mit.
Note: please check DeepKPG for using this model in huggingface, including setting up the newly trained tokenizer.
All-time downloads
1K
Public
Repo size
3.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.5 GB · 100%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- License
- mit
- Model type
- bart
- Access
- Public
- Created
- May 7, 2023
- Updated
- May 7, 2023
- SHA
- ea265322
Model card
From the Hugging Face model README
Note: please check DeepKPG for using this model in huggingface, including setting up the newly trained tokenizer.
Paper: Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study
@article{https://doi.org/10.48550/arxiv.2212.10233,
doi = {10.48550/ARXIV.2212.10233},
url = {https://arxiv.org/abs/2212.10233},
author = {Wu, Di and Ahmad, Wasi Uddin and Chang, Kai-Wei},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
Pre-training Corpus: S2ORC (titles and abstracts)
Pre-training Details:
- Pre-trained from scratch with a science vocabulary
- Batch size: 2048
- Total steps: 250k
- Learning rate: 3e-4
- LR schedule: polynomial with 10k warmup steps
- Masking ratio: 30%, Poisson lambda = 3.5
- Task
- Feature Extraction
- Library
- transformers
- Type
- bart
- License
- mit
- Created
- May 7, 2023
- Updated
- May 7, 2023