gliner-bi-large-v2.0-onnx

GLiNER bi-large v2.0 (ONNX)

ONNX export of knowledgator/gliner-bi-large-v2.0 — the bi-encoder span architecture, which encodes text and entity labels with two separate transformers and matches them with a dot product. Because the label side never sees the text, its embeddings can be computed once for a label set and reused across every document.

As far as we can tell this is the first published ONNX export of any GLiNER bi-encoder. Upstream's convert_to_onnx does not work on this family — see #225 and #237, both open, both dying with AttributeError: 'NoneType' object has no attribute 'max' at text_lengths.max(). That is a bug in the export harness rather than a limitation of the model: it calls GLiNER.forward with a positional tuple, and text_lengths only ever exists to size a padding bound for batches. Exporting the two branches directly never reaches the line.

Exported for SecuPi's unstructured AI service (SPAI), which runs it through ONNX Runtime from Java. Published so that service can fetch it without a PyTorch dependency; anyone else is welcome to use it under the same licence as the base model.

Files

There is no .onnx.data: both graphs stay under the 2 GB protobuf limit.

How to run it

The text side carries no label prompt — that is the point of a bi-encoder. <<ENT>> and <<SEP>> appear in the tokenizer's added vocab but the embedding matrix was never resized for them (from_pretrained declines to resize for bi-encoders), so they are unusable vestiges. Do not emit them.

1. Split the raw text into words with \w+(?:[-_]\w+)*|\S. Do not lowercase — the text encoder is cased. In Java compile this with Pattern.UNICODE_CHARACTER_CLASS; Java's \w is ASCII-only by default, which splits Müller into three words and shifts every later offset.

2. input_ids = [CLS] + concat(subtokens of " " + word) + [SEP], and record word_first_index[w], the absolute index of word w's first sub-token (so [0] == 1). Pooling is first-sub-token; the graph does the gather.

3. Run span_encoder.onnx:

   input	shape
   input_ids	[1, L] int64
   attention_mask	[1, L] int64
   word_first_index	[1, W] int64

   output	shape
   span_rep	[1, W, 12, 768] float32

   span_rep[0][w][k] is the span covering words w through w+k inclusive. The grid is built inside the graph, so there is no flat span list whose ordering you could get wrong. Entries where w + k >= W are invalid and must be skipped.

4. Tokenize the labels with labels_tokenizer/, padded to the batch maximum, and run label_encoder.onnx:

   input	shape
   labels_input_ids	[C, S] int64
   labels_attention_mask	[C, S] int64

   output	shape
   prompts_embedding	[C, 768] float32

   This output depends only on the label text, so cache it per label set.

5. scores[w][k][c] = sigmoid(dot(span_rep[0][w][k], prompts_embedding[c])).

6. Take the argmax label per span; drop anything below the threshold.

7. Sort survivors by score descending and greedily keep a span only if it overlaps nothing already kept — flat NER, across labels, on inclusive ranges.

8. Char offsets are wordCharStart[w] .. wordCharEnd[w+k].

A runnable version of exactly the above, which diffs its output against native PyTorch, ships with the export tooling as verify_gbi2_export.py.

Export notes

• The RNN is dead code on this path. gliner_config.json says has_rnn: true and the checkpoint carries 1.58M trained bidirectional LSTM weights, but BaseBiEncoderModel never calls them — only the uni-encoder and streaming paths do. A forward-hook count confirms zero invocations, and the exporter re-checks it and refuses to run if that ever changes, because a graph that silently omitted a live layer would still return plausible entities.
• SpanMarkerV0 bakes the word count. Its forward ends with .view(B, span_idx.size(1) // max_width, max_width, D), where the middle term is a Python int read off the traced shape — so a graph traced on 37 words rejects every other count. Reshaping that axis with -1 is exactly equivalent and keeps it dynamic.
• ModernBERT did not need a length patch. Its local-attention mask and position ids are built from arange(seq_len), which can trace to a constant, so the export is verified from 4 to 242 tokens rather than assumed. It generalises; no patch was added.
• Traced in the middle of the verification range on purpose: tracing long and verifying short can pass on a baked table by slicing.

Verified against native PyTorch

Before writing anything, the exporter reproduces the model's own logits from just these two graphs' contents — bit-exactly, 0.000e+00, at 3/18/48 words and 1/3/8 labels.

And end to end, ONNX + the decode above returns identical entity sets to model.predict_entities() on 6/6 test documents.

Provenance

Exported from knowledgator/gliner-bi-large-v2.0 at revision 71eec1d5fe57684cd6669ad8a1e1e3e7401a9a55, with gliner==0.2.28, transformers==4.57.0, torch==2.13.0, opset 18.

Text encoder: jhu-clsp/ettin-encoder-400m. Label encoder: BAAI/bge-base-en-v1.5.

Licence

Apache 2.0, inherited from the base model. This repository redistributes the same weights in a different serialisation; all credit for the model belongs to Knowledgator.