muril-ner-multilingual

muril-ner-multilingual 🏷️

This model is a fine-tuned version of google/muril-base-cased for Named Entity Recognition (NER) across three major Indic languages: Marathi (mr), Bengali (bn), and Telugu (te).

It uses a joint multilingual full fine-tuning strategy to extract three primary entity types:

• 👤 PER (Person)
• 🏢 ORG (Organization)
• 📍 LOC (Location)

Model Description

• Developed by: Learner-sai
• Model Type: Token Classification (NER)
• Base Model: google/muril-base-cased
• Languages: Marathi (mr), Bengali (bn), Telugu (te)
• License: Apache 2.0

MuRIL (Multilingual Representations for Indian Languages) was adapted by adding a 7-class linear sequence classification head (O, B-PER, I-PER, B-ORG, I-ORG, B-LOC, I-LOC). All parameters were updated during joint training across all three target languages to leverage cross-lingual transfer.

Intended Uses & Limitations

Intended Uses 🎯

• Automated entity extraction from news articles, social media, and documents in Marathi, Bengali, and Telugu.
• Downstream NLP applications such as information retrieval, question answering, and knowledge graph construction for Indic languages.

Limitations ⚠️

• Grammatical Suffixes: In agglutinative or highly inflected languages like Marathi and Telugu, locative or case suffixes (e.g., Marathi -त in "दिल्लीत") may sometimes be included inside the predicted entity span.
• Entity Scope: The model is trained exclusively on PER, ORG, and LOC tags; it will not recognize other categories like dates, monetary values, or product names.

Training and Evaluation Data

The model was trained on a combined dataset comprising annotated sentences across Marathi, Bengali, and Telugu.

• Label Schema: IOB2 format (B-, I-, O) with 7 total classes.
• Token Alignment: Subword tokenization artifacts were handled using -100 label masking on non-initial subwords to ensure clean cross-entropy loss calculation.

Training Procedure

Training Hyperparameters

• Learning Rate: 3e-05
• Train Batch Size: 16
• Eval Batch Size: 32
• Epochs: 3
• Optimizer: AdamW (fused) with $\beta_1=0.9, \beta_2=0.999, \epsilon=1\text{e-}08$
• LR Scheduler: Linear with 937 warmup steps (~10% of total steps)
• Mixed Precision: Native FP16 (fp16=True)
• Seed: 42

Evaluation Metrics

Evaluated on the validation split using seqeval (entity-level span matching):

Framework Versions

• Transformers: 5.14.1
• PyTorch: 2.11.0+cu128
• Datasets: 2.21.0
• Tokenizers: 0.22.2