roberta-base-ontonotes5-ner

RoBERTa-base fine-tuned on OntoNotes 5.0

This model is a fine-tuned version of FacebookAI/roberta-base on the English subset of the OntoNotes 5.0 (CoNLL-2012) dataset. RoBERTa optimizes the BERT pretraining approach by using dynamic masking, removing next-sentence prediction, and training on larger batches with byte-level BPE.

📊 Performance

The following results were achieved on the OntoNotes 5.0 (v12) test set:

🛠 Training Details

• Architecture: RobertaForTokenClassification
• Tokenizer: RobertaTokenizerFast (with add_prefix_space=True)
• Epochs: 5
• Learning Rate: 2e-5
• Batch Size: 16 per device (Total 32 on 2x V100 GPUs)
• Max Sequence Length: 128
• Weight Decay: 0.01
• Mixed Precision (FP16): Enabled

📂 Project Assets

• GitHub Repository: Learnrr/ontonotes5_ner_evaluation

🚀 Usage

from transformers import pipeline

model_checkpoint = "learnrr/roberta-base-ontonotes5-ner"
token_classifier = pipeline(
    "token-classification", 
    model=model_checkpoint, 
    aggregation_strategy="simple"
)

text = "Microsoft Corporation was founded by Bill Gates and Paul Allen."
results = token_classifier(text)

for entity in results:
    print(f"Entity: {entity['word']} | Label: {entity['entity_group']} | Score: {entity['score']:.4f}")