bilstm-crf-ontonotes5-ner

BiLSTM-CRF for NER (OntoNotes 5.0)

This is a custom BiLSTM-CRF model fine-tuned on the English subset of the OntoNotes 5.0 (CoNLL-2012) dataset. Unlike Transformer-based models, this architecture combines the sequential feature extraction of BiLSTMs with the structural inference of a Conditional Random Field (CRF) layer, initialized with pre-trained GloVe 300d word embeddings.

📊 Performance

The model was evaluated on the OntoNotes 5.0 (v12) official test set using seqeval:

🛠 Model Architecture

• Embedding Layer: GloVe 300d (wiki-gigaword), fine-tuned during training.
• Encoder: 2-layer Bi-directional LSTM with 512 hidden units.
• Decoder: Linear-chain CRF for optimal tag sequence decoding.
• Dropout: 0.5 (Applied to embeddings and LSTM outputs).

📂 Project Assets

• GitHub Repository: Learnrr/ontonotes5_ner_evaluation

📂 Training Infrastructure

• Framework: PyTorch with DistributedDataParallel (DDP).
• Hardware: Multi-GPU (NVIDIA V100) setup with NCCL backend.
• Hyperparameters:
  • Optimizer: AdamW (lr=1e-3, weight_decay=0.01)
  • Scheduler: Linear warmup with decay (warmup_ratio=0.1)
  • Epochs: 20
  • Batch Size: 32 per GPU (Effective batch size 64)
  • Max Length: 128 tokens

🚀 Usage

import torch
from model import BiLSTM_CRF # Ensure class definition is accessible

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# 1. Load mappings
vocab = torch.load("vocab.pth")
label_list = torch.load("label_list.pth")

# 2. Initialize and Load Weights
model = BiLSTM_CRF(
    v_size=len(vocab), 
    t_size=len(label_list), 
    e_dim=300, 
    h_dim=512, 
    w_matrix=torch.zeros(len(vocab), 300)
)

state_dict = torch.load("best_bilstm_crf_ddp.pth", map_location=device)
# Standardize keys (remove 'module.' from DDP training)
new_state_dict = {k.replace('module.', ''): v for k, v in state_dict.items()}
model.load_state_dict(new_state_dict)
model.to(device).eval()