Downloads · 30 days
33.5K
3% of all-time downloads
finbert-pretrain
yiyanghkust/finbert-pretrain
finbert-pretrain is a fill-mask model from yiyanghkust. Use it when you need the model to fill a missing word. It is set up for transformers.
FinBERT is a BERT model pre-trained on financial communication text. The purpose is to enhance financial NLP research and practice. It is trained on the following three financial communication corpus. The total corpor…
All-time downloads
1.2M
Public
Repo size
1.4 GB
Likes
40
Public
Repo makeup
Click a slice to open those files.
.bin442 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Oct 17, 2022
- SHA
- 88ab954a
Try a prompt
Model card
From the Hugging Face model README
FinBERT is a BERT model pre-trained on financial communication text. The purpose is to enhance financial NLP research and practice. It is trained on the following three financial communication corpus. The total corpora size is 4.9B tokens.
- Corporate Reports 10-K & 10-Q: 2.5B tokens
- Earnings Call Transcripts: 1.3B tokens
- Analyst Reports: 1.1B tokens
If you use the model in your academic work, please cite the following papers:
Huang, Allen H., Hui Wang, and Yi Yang. "FinBERT: A Large Language Model for Extracting Information from Financial Text." Contemporary Accounting Research (2022).
Yang, Yi, Mark Christopher Siy Uy, and Allen Huang. "Finbert: A pretrained language model for financial communications." arXiv preprint arXiv:2006.08097 (2020).
FinBERT can be further fine-tuned on downstream tasks. Specifically, we have fine-tuned FinBERT for financial sentiment analysis, ESG classification, Forward-looking statement classification and etc. Visit FinBERT.AI for more details on these task-specific models and recent development of FinBERT.
- Task
- Fill-Mask
- Library
- transformers
- Created
- Mar 2, 2022
- Updated
- Oct 17, 2022