distilroberta-ConLL2003

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

Model Description

This model is a fine-tuned version of distilroberta-base on ConLL2003 dataset. It achieves the following results on the evaluation set in Named Entity Recognition (NER)/Token Classification task:

• Loss: 0.0585
• F1: 0.9536

Model Performance

• 1st Place: This fine-tuned model is topped on the best scores ( F1: 94.6%) from Named Entity Recognition (NER) on CoNLL 2003 (English).
• 6th Place: This fine-tuned model is ranked in the 6th place from the Token Classification on conll2003 leaderboard

Model Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("jinhybr/distilroberta-ConLL2003")
model = AutoModelForTokenClassification.from_pretrained("jinhybr/distilroberta-ConLL2003")

nlp = pipeline("ner", model=model, tokenizer=tokenizer, grouped_entities=True)
example = "My name is Tao Jin and live in Canada"
ner_results = nlp(example)
print(ner_results)

[{'entity_group': 'PER', 'score': 0.99686015, 'word': ' Tao Jin', 'start': 11, 'end': 18}, {'entity_group': 'LOC', 'score': 0.9996836, 'word': ' Canada', 'start': 31, 'end': 37}]

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 24
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6.0

Training results

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.17.0
• Tokenizers 0.15.1