metaextractor-spacy

<img src="https://huggingface.co/finding-fossils/metaextractor/resolve/main/ffossils-logo-text.png" width="400">

Finding Fossils - SpaCy Transformer

<!-- Provide a quick summary of what the model is/does. -->

This model extracts metadata from research articles related to Paleoecology.

The entities detected by this model are:

• AGE: when historical ages are mentioned such as 1234 AD or 4567 BP (before present)
• TAXA: plant or animal taxa names indicating what samples contained
• GEOG: geographic coordinates indicating where samples were excavated from, e.g. 12'34"N 34'23"W
• SITE: site names for where samples were excavated from
• REGION: more general regions to provide context for where sites are located
• EMAIL: researcher emails in the articles able to be used for follow-up contact
• ALTI: altitudes of sites from where samples were excavated, e.g. 123 m a.s.l (above sea level)

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

• Developed by: Ty Andrews, Jenit Jain, Shaun Hutchinson, Kelly Wu, and Simon Goring
• Shared by: Neotoma Paleocology Database
• Model type: Token Classification
• Language(s) (NLP): English
• License: MIT
• Text Embeddings: roberta-base
• Named Entity Recognition: spacy transition-based S-LSTMs.

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/NeotomaDB/MetaExtractor
• Paper: https://arxiv.org/pdf/1603.01360.pdf

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

This model can be used to extract entities from any text that are Paeleoecology related or tangential. Potential uses include identifying unique SITE names in research papers in other domains.

Direct Use

This model is deployed on the xDD (formerly GeoDeepDive) servers where it is getting fed new research articles relevant to Neotoma and returning the extracted data.

This approach could be adapted to other domains by using the training and development code found github.com/NeotomaDB/MetaExtractor to run similar data extraction for other research domains.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

This model was trained entirely on English research articles and will likely not perform well on research in other languages. Also, the articles used to train the model were chosen based on being already present in the Neotoma database and therefore may have selection bias as they represent what is already known to be relevant to Neotoma and may not correctly manage new, previously missed articles.

How to Get Started with the Model

Use the code below to get started with the model.

pip install https://huggingface.co/finding-fossils/metaextractor-spacy/resolve/main/en_metaextractor_spacy-any-py3-none-any.whl

# Using spacy.load().
import spacy
nlp = spacy.load("en_metaextractor_spacy")

# Importing as module.
import en_metaextractor_spacy
ner_pipe = en_metaextractor_spacy.load()

doc = ner_pipe("In Northern Canada, the BGC site core was primarily made up of Pinus pollen.")

entities = []

for ent in doc.ents:
  entities.append({
      "start": ent.start_char,
      "end": ent.end_char,
      "labels": [ent.label_],
      "text": ent.text
  })

print(entities)

# Output
[
  {
    "start": 3,
    "end": 19,
    "labels": ["REGION"],
    "text": " Northern Canada,",
  },
  {
    "start": 24,
    "end": 27,
    "labels": ["SITE"],
    "text": " BGC",
  },
  {
    "start": 63,
    "end": 68,
    "labels": ["TAXA"],
    "text": " Pinus",
  }
]

Training Details

Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

The model was trained using a set of 39 research articles deemed relevant to the Neotoma Database. All articles were written in English. The entities were labeled by the project team along with using pre-labelling with early models to speed up the labelling process.

A 70/15/15 train/val/test split was used which had the following breakdown of words and entities.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

For full training details please see the GitHub repository and Wiki: github.com/NeotomaDB/MetaExtractor

Results & Metrics

For full model results see the report here: Final Project Report