Downloads · 30 days
9
1% of all-time downloads
Fin-RoBERTa
SUFEHeisenberg/Fin-RoBERTa
Fin-RoBERTa is a text classification model from SUFEHeisenberg. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
We collects financial domain terms from Investopedia's Financia terms dictionary, NYSSCPA's accounting terminology guide and Harvey's Hypertextual Finance Glossary to expand RoBERTa's vocab dict.
All-time downloads
1.1K
Public
Repo size
1 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.bin505 MB · 99%
At a glance
- Task
- Text Classification
- Library
- transformers
- License
- apache-2.0
- Model type
- roberta
- Access
- Public
- Created
- Feb 23, 2024
- Updated
- Feb 23, 2024
- SHA
- ccdf8ff1
Try a prompt
Model card
From the Hugging Face model README
We collects financial domain terms from Investopedia's Financia terms dictionary, NYSSCPA's accounting terminology guide and Harvey's Hypertextual Finance Glossary to expand RoBERTa's vocab dict.
Based on added-financial-terms RoBERTa, we pretrained our model on multilple financial corpus:
- Financial Terms
- Financial Datasets
- Earnings Call 2016-2023 NASDAQ 100 components stocks's Earnings Call Transcripts.
In continual pretraining step, we apply following experiments settings to achieve better finetuned results on Four Financial Datasets:
- Masking Probability: 0.4 (instead of default 0.15)
- Warmup Steps: 0 (deriving better results than models with warmup steps)
- Epochs: 1 (is enough in case of overfitting)
- weight_decay: 0.01
- Train Batch Size: 64
- FP16
- Task
- Text Classification
- Library
- transformers
- Type
- roberta
- License
- apache-2.0
- Languages
- en
- Created
- Feb 23, 2024
- Updated
- Feb 23, 2024