sec-bert-finer-ord-ner

sec-bert-finer-ord-ner

A token-classification (NER) fine-tune of nlpaueb/sec-bert-base on the FiNER-ORD dataset, for extracting person (PER), location (LOC), and organization (ORG) entities from financial/SEC-filing-style English text.

This is an academic exercise: a small pipeline that runs financial-news NLP (sentiment + NER) over a corpus of company news articles.

Intended use

• Named entity recognition (PER/LOC/ORG) on financial news, earnings-call transcripts, and SEC-filing-style English text.
• Non-commercial use only — see License below.

Not intended for: general-domain NER (it's tuned for financial text), languages other than English, or any production/commercial deployment.

Label scheme

BIO tagging over three entity types:

Training data

FiNER-ORD (gtfintechlab/finer-ord), a manually annotated financial NER dataset. Token-level rows were regrouped into sentences (grouped by doc_idx/sent_idx) and labels aligned to WordPiece subwords, keeping only the first subword of each token labeled (other subwords set to -100, ignored in the loss).

Training procedure

Fine-tuned from nlpaueb/sec-bert-base with Hugging Face Trainer:

• learning rate: 3e-5
• batch size: 16 (train) / 32 (eval)
• epochs: 8
• weight decay: 0.01
• mixed precision (fp16)
• load_best_model_at_end=True, selected by validation F1
• seed: 42

Evaluation results

Metrics on the FiNER-ORD test split, using the best-validation-F1 checkpoint (selected during training):

(Best validation-split F1 during training: 0.820.)

Downstream pipeline evaluation (real financial news, LLM-judge)

The numbers above are the FiNER-ORD test split. Separately, this checkpoint is run in production inside portfolio-nlp — a pipeline that chunks full financial-news articles (not FiNER-ORD's short annotated sentences), runs this model over every chunk, and merges per-chunk BIO predictions back into article-level entity spans (see "Recommended pre/post-processing" below). That pipeline's accuracy is tracked separately via an LLM-as-judge eval (docs/evaluation.md in the repo above), most recently at n=8000 real articles (2026-09-12, post word-boundary-merge fix):

¹ share of predicted spans an LLM judge, shown the full source article, couldn't confirm as a real entity.

These are not a substitute for the FiNER-ORD test-split numbers above — different data (real news vs. held-out annotated sentences), different scoring (LLM judge vs. exact-match seqeval), and they depend on the downstream pipeline's chunking/merge logic, not just this checkpoint in isolation. Included as a second, real-world-traffic data point.

Recommended pre/post-processing

Chunk long documents; don't truncate. This model's max sequence length is BERT-base-sized (512 tokens), well short of a typical news article or filing. Split on sentence/paragraph boundaries, run each chunk separately, and re-offset each chunk's character spans back into full-document coordinates when merging — truncating instead silently drops every entity past the cutoff.

If you aggregate raw per-token BIO predictions yourself (rather than transformers' built-in aggregation_strategy), merge spans word-boundary- aware: only a word's first WordPiece subword should open, close, or redirect a span; a continuation subword (same word) should only ever extend whatever its word's first subword already decided, never be read independently. This matches how the model was trained — per "Training data" above, only each word's first subword carries a label; continuation subwords get -100 and zero training signal, so their raw argmax is unreliable on its own. A production deployment of this model that merged by raw per-token argmax (subword-naive) produced real, systematic artifacts: bare-digit spans like "3" tagged ORG, split off "3M" — 1,859 such "3"-only entities in one 17.6M-row NER table, 62% of a wider single-digit- ORG pattern, plus the same shape off "3Q24", "1MDB", "L3Harris", and "Boeing's 777 jet". If using the transformers pipeline, prefer aggregation_strategy="first" over "simple"/"average" — "first" uses each word's first-subword label, the same rule the model was trained under; the other strategies can let an undertrained continuation subword's score pull a span's boundary or label. A defensive minimum-entity-length filter (reject 1-character spans) is a reasonable last-resort net regardless of which merge approach you use.

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

tokenizer = AutoTokenizer.from_pretrained("gamug/sec-bert-finer-ord-ner")
model = AutoModelForTokenClassification.from_pretrained("gamug/sec-bert-finer-ord-ner")

# aggregation_strategy="first" matches how this model was trained (only each
# word's first WordPiece subword carries a label) — see "Recommended
# pre/post-processing" above.
ner = pipeline("ner", model=model, tokenizer=tokenizer, aggregation_strategy="first")
ner("3M Company (NYSE:MMM) reported results with CEO Mike Roman on the call.")

License

This model is a derivative of two upstream works with different licenses:

• Base model nlpaueb/sec-bert-base: CC-BY-SA-4.0
• Training data gtfintechlab/finer-ord: CC-BY-NC-4.0

Because the training data is non-commercial-only, this fine-tuned model is released under CC-BY-NC-4.0: attribution required, non-commercial use only. Please credit both upstream works (linked above) if you use or build on this model.