matscibert-chem-ner

MatSciBERT — Chemical Named Entity Recognition

Fine-tuned m3rg-iitd/matscibert on the CHEMDNER corpus for chemical named entity recognition (NER) in biomedical and scientific text.

The model identifies and highlights chemical compound names, drug names, and chemical formulas in free text.

---

Model Description

---

Labels

After aggregation, the pipeline outputs CHEM spans combining B-CHEM/I-CHEM tokens.

---

Evaluation Results

Evaluated on the CHEMDNER validation set (~6 808 examples) using seqeval (entity-span level):

---

Usage

With pipeline

from transformers import pipeline

ner = pipeline(
    "ner",
    model="teman67/matscibert-chem-ner",
    aggregation_strategy="simple",
)

text = "Nitric oxide reacts with oxygen to form nitrogen dioxide."
results = ner(text)

for entity in results:
    print(f"{entity['word']:<25} {entity['entity_group']}  ({entity['score']:.1%})")

Output:

nitric oxide              CHEM  (100.0%)
oxygen                    CHEM  (100.0%)
nitrogen dioxide          CHEM  (99.9%)

With AutoModel

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("teman67/matscibert-chem-ner")
model = AutoModelForTokenClassification.from_pretrained("teman67/matscibert-chem-ner")

inputs = tokenizer("Aspirin inhibits COX-1 and COX-2 enzymes.", return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

predictions = outputs.logits.argmax(-1)
tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
labels = [model.config.id2label[p.item()] for p in predictions[0]]

for token, label in zip(tokens, labels):
    if not token.startswith("##") and token not in ("[CLS]", "[SEP]"):
        print(f"{token:<20} {label}")

---

Training Details

Training data split from the CHEMDNER corpus:

---

Training Code

Source code available at: github.com/teman67/Fine-tuning-Materials-Scientific-NER-

---

Intended Use & Limitations

Intended for:

• Extracting chemical and drug names from biomedical literature
• Pre-processing step for downstream chemistry/materials science NLP tasks
• Scientific text mining pipelines

Limitations:

• Trained on PubMed/patent text — performance may degrade on very different domains
• Recognises a single entity type (CHEM); does not distinguish subtypes (drugs, elements, formulas, etc.)
• Sentences longer than 128 WordPiece tokens are truncated — chemicals at the end of long passages may be missed
• Entity strings matched by surface form; ambiguous terms (e.g. Mercury the planet vs element) are always tagged as CHEM

---

References

• MatSciBERT: Gupta et al., "MatSciBERT: A materials domain language model for text mining and information extraction", npj Computational Materials, 2022. doi:10.1038/s41524-022-00784-w
• CHEMDNER: Krallinger et al., "The CHEMDNER corpus of chemicals and drugs and its annotation principles", Journal of Cheminformatics, 2015. doi:10.1186/1758-2946-7-S1-S2

---

License

MIT