Downloads · 30 days
12
1% of all-time downloads
newsbert
uclanlp/newsbert
newsbert is a fill-mask model from uclanlp. Use it when you need the model to fill a missing word. It is set up for transformers.
Paper: Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study
All-time downloads
1.3K
Public
Repo size
1.4 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.h5534 MB · 55%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- bert
- Access
- Public
- Created
- Dec 16, 2022
- Updated
- Dec 28, 2022
- SHA
- a26bfefc
Try a prompt
Model card
From the Hugging Face model README
Paper: Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study
@article{https://doi.org/10.48550/arxiv.2212.10233,
doi = {10.48550/ARXIV.2212.10233},
url = {https://arxiv.org/abs/2212.10233},
author = {Wu, Di and Ahmad, Wasi Uddin and Chang, Kai-Wei},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
Pre-training Corpus: RealNews
Pre-training Details:
- Resume from bert-base-uncased
- Batch size: 512
- Total steps: 250k
- Learning rate: 1e-4
- LR schedule: linear with 4k warmup steps
- Masking ratio: 15% dynamic masking
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- Created
- Dec 16, 2022
- Updated
- Dec 28, 2022