spanbert-base-finetuned-tacred

mrm8488/spanbert-base-finetuned-tacred

spanbert-base-finetuned-tacred is a feature extraction model from mrm8488. Use it when you need embeddings to search or compare text. It is set up for transformers.

SpanBERT created by Facebook Research and fine-tuned on TACRED dataset by them

Downloads · 30 days

1% of all-time downloads

All-time downloads

1.8K

Public

Repo size

4 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin1.3 GB · 50%

At a glance

Task: Feature Extraction
Library: transformers
Model type: bert
Access: Public
Created: Mar 2, 2022
Updated: May 20, 2021
SHA: 52aefc31

Model card

From the Hugging Face model README

SpanBERT base fine-tuned on TACRED

SpanBERT created by Facebook Research and fine-tuned on TACRED dataset by them

Details of SpanBERT

SpanBERT: Improving Pre-training by Representing and Predicting Spans

Dataset 📚

TACRED A large-scale relation extraction dataset with 106k+ examples over 42 TAC KBP relation types.

Model fine-tuning 🏋️‍

You can get the fine-tuning script here

python code/run_tacred.py \
  --do_train \
  --do_eval \
  --data_dir <TACRED_DATA_DIR> \
  --model spanbert-base-cased \
  --train_batch_size 32 \
  --eval_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 10 \
  --max_seq_length 128 \
  --output_dir tacred_dir \
  --fp16

Results Comparison 📝

	SQuAD 1.1	SQuAD 2.0	Coref	TACRED
	F1	F1	avg. F1	F1
BERT (base)	88.5*	76.5*	73.1	67.7
SpanBERT (base)	92.4*	83.6*	77.4	68.2 (this one)
BERT (large)	91.3	83.3	77.1	66.4
SpanBERT (large)	94.6	88.7	79.6	70.8

Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers.

Created by Manuel Romero/@mrm8488

Made with <span style="color: #e25555;">♥</span> in Spain

Task: Feature Extraction
Library: transformers
Type: bert
Languages: en
Created: Mar 2, 2022
Updated: May 20, 2021