span-marker-bert-base-uncased-bionlp

SpanMarker with bert-base-uncased on BioNLP2004

This is a SpanMarker model trained on the BioNLP2004 dataset that can be used for Named Entity Recognition. This SpanMarker model uses bert-base-uncased as the underlying encoder. See train.py for the training script.

Model Details

Model Description

• Model Type: SpanMarker
• Encoder: bert-base-uncased
• Maximum Sequence Length: 256 tokens
• Maximum Entity Length: 8 words
• Training Dataset: BioNLP2004
• Language: en
• License: other

Model Sources

• Repository: SpanMarker on GitHub
• Thesis: SpanMarker For Named Entity Recognition

Model Labels

Evaluation

Metrics

Uses

Direct Use

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-uncased-bionlp")
# Run inference
entities = model.predict("In erythroid cells most of the transcription activity was contained in a 150 bp promoter fragment with binding sites for transcription factors AP2, Sp1 and the erythroid-specific GATA-1.")

Downstream Use

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-uncased-bionlp")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("tomaarsen/span-marker-bert-base-uncased-bionlp-finetuned")

</details>

Training Details

Training Set Metrics

Training Hyperparameters

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3

Training Results

Environmental Impact

Carbon emissions were measured using CodeCarbon.

• Carbon Emitted: 0.045 kg of CO2
• Hours Used: 0.296 hours

Training Hardware

• On Cloud: No
• GPU Model: 1 x NVIDIA GeForce RTX 3090
• CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
• RAM Size: 31.78 GB

Framework Versions

• Python: 3.9.16
• SpanMarker: 1.3.1.dev
• Transformers : 4.29.2
• PyTorch: 2.0.1+cu118
• Datasets: 2.14.3
• Tokenizers: 0.13.2