Downloads · 30 days
105
2% of all-time downloads
spanbert-large-finetuned-tacred
mrm8488/spanbert-large-finetuned-tacred
spanbert-large-finetuned-tacred is a feature extraction model from mrm8488. Use it when you need embeddings to search or compare text. It is set up for transformers.
SpanBERT created by Facebook Research and fine-tuned on TACRED dataset by them
All-time downloads
5.7K
Public
Parameters
334M
4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.3 GB · 33%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- Model type
- bert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- May 7, 2023
- SHA
- 6ea99387
Model card
From the Hugging Face model README
SpanBERT large fine-tuned on TACRED
SpanBERT created by Facebook Research and fine-tuned on TACRED dataset by them
Details of SpanBERT
SpanBERT: Improving Pre-training by Representing and Predicting Spans
Dataset 📚
TACRED A large-scale relation extraction dataset with 106k+ examples over 42 TAC KBP relation types.
Model fine-tuning 🏋️
You can get the fine-tuning script here
python code/run_tacred.py \
--do_train \
--do_eval \
--data_dir <TACRED_DATA_DIR> \
--model spanbert-large-cased \
--train_batch_size 32 \
--eval_batch_size 32 \
--learning_rate 2e-5 \
--num_train_epochs 10 \
--max_seq_length 128 \
--output_dir tacred_dir \
--fp16
Results Comparison 📝
| SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED | |
|---|---|---|---|---|
| F1 | F1 | avg. F1 | F1 | |
| BERT (base) | 88.5* | 76.5* | 73.1 | 67.7 |
| SpanBERT (base) | 92.4* | 83.6* | 77.4 | 68.2 |
| BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 |
| SpanBERT (large) | 94.6 | 88.7 | 79.6 | 70.8 (this one) |
Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers.
Created by Manuel Romero/@mrm8488
Made with <span style="color: #e25555;">♥</span> in Spain
- Task
- Feature Extraction
- Library
- transformers
- Type
- bert
- Languages
- en
- Created
- Mar 2, 2022
- Updated
- May 7, 2023