otter-bi-mmbert

🦦 otter-bi-mmbert

Otter is a multilingual, open-type named entity recognizer. You give it a piece of text and a list of entity types in plain language -- ["person", "band", "chemical compound"] -- and it returns the character spans of the entities of those types. There is no fixed label set and no fine-tuning step: the types are part of the input.

Bi-encoder. The text and the entity type names are encoded separately, and candidate spans are scored against the label embeddings. Label embeddings depend only on the label set, so they can be computed once and reused across a whole corpus -- the cheaper option when the same types are applied to many inputs.

• Text encoder: jhu-clsp/mmBERT-base
• Label encoder: google-bert/bert-base-multilingual-uncased
• Max sequence length: 1024 tokens
• Max span length: 30 tokens

Usage

from transformers import AutoModel

model = AutoModel.from_pretrained("whoisjones/otter-bi-mmbert", trust_remote_code=True)
model.eval()

entities = model.predict(
    "Angela Merkel besuchte gestern das Brandenburger Tor in Berlin.",
    labels=['person', 'organization', 'location'],
)

for entity in entities:
    print(f"{entity['text']!r:25} {entity['label']:15} {entity['score']:.2f}")

'Angela Merkel'           person          0.84
'Brandenburger Tor'       location        0.30

Each entity is a dict with text, label, start, end (character offsets into the input string) and score. Pass a list of strings to run on a batch; you then get one list of entities per input, in the same order:

model = model.to("cuda")

texts = ["Angela Merkel besuchte das Brandenburger Tor.", "Sony was founded in Tokyo."]
results = model.predict(texts, labels=["person", "organization", "location"], batch_size=16)

Threshold

predict keeps spans scoring above threshold, which defaults to config.prediction_threshold (0.2 for this checkpoint, chosen by calibrating macro-F1 across the evaluation suite). Lower it for higher recall, raise it for higher precision:

entities = model.predict(text, labels=labels, threshold=0.1)

Because the label set is part of the input, the useful threshold shifts with how many types you ask for and how specific they are. If you have a few hundred annotated sentences from your own domain, re-calibrating on those is worth more than any default.

Writing good label names

The label is read as natural language, so it carries meaning. "politician" and "person" select different spans, and a phrase like "chemical compound" works as well as a single word. Prefer the wording you would use to describe the type to a person.

Reusing label embeddings

The type encoder only sees the label names, so its output can be computed once and reused for every batch:

type_inputs = model.encode_labels(labels)   # do this once

predict does this per call; drop down to forward if you are running over a large corpus with a fixed label set.

Fine-tuning

collate_fn.py in this repository holds the training and evaluation collators. See the GitHub repository for the full training pipeline, the evaluation suite, and the data preparation scripts.

Model family

The cross-encoders are the stronger models; the bi-encoders are cheaper when one label set is applied across a large corpus.

License

Apache 2.0.