goby-9-ie-vi

goby-9-ie-vi

Joint entity + relation extraction for Vietnamese. The best-calibrated model in the goby family: higher precision at the same recall than its siblings. On clean text at each model own best threshold it ties goby-4 rather than beating it (see Measurement correction); on degraded text it is the worst of the family. Pick it for well-formed prose where you care about precision.

Fine-tuned from knowledgator/gliner-relex-large-v1.0. Released by Belumind. Apache-2.0.

Try it in the browser - no install, runs on ZeroGPU.

Which goby do I want?

goby-9 is not robust to degraded text - see Known limitations. That is not a small caveat, read it before choosing.

What changed

Same training data as goby-4. The only difference is the loss configuration: negative sampling ratio 1.0 -> 2.5, focal loss alpha 0.75 -> 0.5. No new data, no augmentation.

This matters because it moves the precision/recall curve rather than sliding along it:

If the two models only differed by an operating point, precision at matched recall would be identical. It is not: +3.9 and +4.1 in the range most people run at.

Measurement correction (2026-09-21)

Every NER number in this card was originally measured with a placeholder relation label (relations=["x"]) - a setting nobody uses in production. Re-measured with the real relation list, same benchmark, one shared harness, every checkpoint scores higher, and the gain grows with how much the model was fine-tuned:

The base model barely moves (+0.9); goby-9 gains +4.6. The relation head learned during fine-tuning feeds back into entity detection, so passing the relations you actually want is worth several F1 on its own - pass them even if you only need NER.

At each model best threshold, goby-9 and goby-4 are INDISTINGUISHABLE: 71.6 vs 71.4, difference +0.22 F1, 95% bootstrap CI [-0.79, +1.29], 68% of resamples positive. Comparisons at one shared threshold favour goby-9; comparisons at each model own operating point do not.

The per-type and relation tables below have not been re-measured and still reflect the placeholder setting.

Results

Benchmark: 450 held-out Vietnamese sentences, 1,328 hand-audited entity spans. All figures at threshold=0.5 unless stated.

Typed NER

Paired bootstrap at a shared threshold, goby-9 minus goby-4: +3.8 F1, 95% CI [+2.7, +4.8], 100% of resamples positive. At each model own best threshold with real relation labels the two are indistinguishable (+0.22, CI [-0.79, +1.29]) - see Measurement correction above.

Threshold sweep:

It also fires far less: 1,672 predictions at 0.5 against goby-4 at 2,005, for higher recall-adjusted precision.

Per entity type (F1)

goby-9 wins on 7 of 10. It loses on tac pham (52.6 -> 33.3), san pham (59.0 -> 51.1) and su kien (40.0 -> 37.5) - the sharper decision boundary costs recall on the types with the fewest training examples.

Relations

Relation quality is essentially unchanged (-1.0 partial, +0.9 strict).

Relations: the operating point matters twice

A relation can only be found if BOTH of its endpoints are detected, so the entity threshold silently caps relation recall. The entity threshold that is best for NER is not the one that is best for relations - they have to be swept together.

Best relation F1 for this model, sweeping both thresholds on 450 sentences / 428 gold relations:

Same benchmark, every model at its own best setting (strict F1): goby-4 59.20 | goby-7 56.97 | goby-9 60.50 | goby-12 47.03 | barb-1 63.41.

Earlier versions of this card reported relations measured at the NER-optimal entity threshold only, which understated them.

Usage

pip install gliner==0.2.29

from gliner import GLiNER

model = GLiNER.from_pretrained("belumind/goby-9-ie-vi").to("cuda")

KEYS = {
    "to_chuc": "tổ chức",   "van_ban": "văn bản",
    "nguoi": "người",       "dia_diem": "địa điểm",
    "ngay": "ngày",         "chuc_vu": "chức vụ",
}
INV = {v: k for k, v in KEYS.items()}

ents, rels = model.inference(
    texts=["Công ty Cổ phần Giải trí Galaxy được thành lập ngày 13 / 9 / 2003 ."],
    labels=list(KEYS.values()),
    relations=["thành lập ngày", "đặt trụ sở tại"],
    threshold=0.5, relation_threshold=0.5,
    return_relations=True, flat_ner=True,
)
out = [{"text": x["text"], "type": INV[x["label"]]} for x in ents[0]]

Pass the Vietnamese labels with diacritics. GLiNER encodes the label string with its text encoder, so to_chuc is effectively an unseen label: it costs about 25 F1. Map to snake_case keys after inference, as above.

Labels must keep their Vietnamese diacritics

GLiNER encodes the label STRING with the same text encoder as the input, so a de-accented label is a different vector. Measured on goby-7, threshold 0.5:

Re-measured on goby-13 at threshold 0.65 with real relation labels. These are TWO separate penalties: dropping the diacritics costs 35 F1, and replacing the space with an underscore costs another 10 on top of that. An earlier version of this table, measured with a placeholder relation label, reported -21.1 and -24.9 - it understated the damage by about half.

snake_case is perfectly fine as an OUTPUT key - goby_labels maps it for you, with to_model_labels(keys) on the way in and extract(..., json_keys=True) on the way out. It just must never be what you hand the model.

About 25 F1, lost with no error and no warning. This repo ships goby_labels.py so the mistake fails loudly instead of silently:

import os, sys
from huggingface_hub import hf_hub_download

sys.path.insert(0, os.path.dirname(hf_hub_download("REPO_ID", "goby_labels.py")))
from goby_labels import extract, check_labels, LabelError

ents, rels = extract(model, [text], json_keys=True)

extract() calls the model with the Vietnamese labels and renames the types to to_chuc, van_ban, ... only in the RETURNED dicts - free, and JSON-safe keys. Passing a de-accented label raises LabelError; an unknown ASCII label warns. check_labels(labels) is the bare assertion if you keep your own inference call.

Weights ship as both model.safetensors (what gliner loads by preference) and pytorch_model.bin; the two are byte-identical, tensor for tensor.

Known limitations

• Degraded text destroys this model. Relation F1 on the same 267 sentences: clean 56.7, without diacritics 6.0, ALL CAPS 6.8. goby-7 scores 28.5 and 30.0 on the same inputs. If your text is not clean and correctly cased, use goby-7, not this.
• chuc vu (job titles): do not trust the 31.2 above. On a separate benchmark of 55 hand-annotated, correctly-delimited job titles, goby-9 gets only 5 exactly right (F1 13.3) - and the base model beats every model in this family there (13 exact, F1 18.3). Our fine-tuning data carries a systematic truncation of Vietnamese job titles, and every goby model inherited it. We publish this because the main benchmark hides it.
• tac pham regressed against goby-4 (52.6 -> 33.3).
• ky (signed document) and ke nhiem (successor of) remain the weakest relations.
• Trained on Vietnamese Wikipedia-style prose. Legal, medical and conversational text are out of distribution.

Citation

@misc{goby9ievi2026,
  title  = {goby-9-ie-vi: a recalibrated joint IE model for Vietnamese},
  author = {Belumind},
  year   = {2026},
  url    = {https://huggingface.co/belumind/goby-9-ie-vi}
}