gliner-small-silver-onnx-cuda

GLiNER Small Silver ONNX CUDA

Final public deployment artifact for English news named-entity recognition with three output classes: persons, organizations, and locations. It contains the fine-tuned GLiNER checkpoint and its FP32 CUDA ONNX export. Anyone may download this public repository without authentication.

Dataset and split conditions

The source data was scraped English news content. It was cleaned to recover article text, exclude unrecoverable/non-English records, remove duplicate leakage, and verify label strings against the cleaned text. The result is a Silver, not manually span-annotated, corpus. Original character offsets were unreliable and are not used as evaluation gold.

The fixed, document-level split is:

Training uses the safe spans. Development selects thresholds. The untouched holdout is reserved for the final comparison. Evaluation uses normalized document-level (entity type, entity string) membership, with exact entity-string F1 for PER, ORG, and LOC; it does not score the unreliable original offsets.

Fine-tuned model quality

Base model: gliner-community/gliner_small-v2.5. Fine-tuning threshold: 0.35. The final untouched-holdout macro F1 is 0.694, an absolute improvement of +0.141 over the original GLiNER-small baseline.

Reproducible release configuration

Training run: 66829308d2a7abf5241012ff83f1bfd2444346df3208443d55ba493eed7de457
PyTorch checkpoint SHA-256: 22076f835e4834d6617022902d1bcb973c54e30702d822578a9d09e42e984ed2
ONNX graph SHA-256: 6db50bd8cacdfe740acd29c3e9385861514a8786e28783aec67ebfc6a39903a8

Training settings used for this run (the best validation-loss checkpoint was retained):

{
  "base_model": "gliner-community/gliner_small-v2.5",
  "best_checkpoint": "checkpoint-390",
  "best_global_step": 390,
  "best_validation_loss": 2.5282070636749268,
  "completed_epochs": 7.0,
  "eval_strategy": "epoch",
  "fp16": true,
  "gradient_accumulation_steps": 16,
  "greater_is_better": false,
  "learning_rate": 1e-05,
  "max_steps": -1,
  "metric_for_best_model": "eval_loss",
  "num_train_epochs": 7,
  "per_device_eval_batch_size": 1,
  "per_device_train_batch_size": 1,
  "save_strategy": "epoch",
  "save_total_limit": 1,
  "trained_in_this_run": true
}

Inference settings inherited by ONNX and Docker:

{
  "batch_size": 4,
  "labels": [
    "person name",
    "organization or institution",
    "city or country location"
  ],
  "overlap_chars": 160,
  "threshold": 0.35,
  "window_chars": 800
}

ONNX CUDA validation

onnx/model.onnx is an FP32 graph validated with onnxruntime-gpu and CUDAExecutionProvider on an NVIDIA GTX 1660 Ti (6 GiB). On the fixed 50-document / 256,453-character Silver validation sample, it processed 25,278 characters/second, compared with 18,588 characters/second for native BF16 PyTorch (1.36x).

The ONNX and native outputs had entity-agreement F1 of 0.9981. ONNX Silver macro F1 was 0.705 versus 0.705 for native BF16 (delta -0.0006). Minor numerical differences are expected between BF16 PyTorch and FP32 ONNX.

A historical 20-user, two-minute Locust infrastructure run completed 79 /predict requests with 0 failures: median latency 6.7 s, p95 29 s, and 0.686 requests/second. Because model weights and thresholds can change between releases, this is retained as an API-concurrency reference rather than claimed as a measurement of this exact release. End-to-end concurrency includes queueing, preprocessing, and long input documents; it is not equivalent to isolated model throughput.

Use

Install the project dependencies, download this repository at a pinned commit, and place it at models/fine-tune_model/gliner_small_silver/. deployment_config.json is part of the artifact and is the authoritative source for the model key, threshold, labels, and chunking settings. Docker Compose reads it automatically; no inference threshold needs to be set manually.

Limitations

This is an English-only model. Silver labels and scraped-news source text introduce dataset limitations. Validate the model against the intended production distribution before using it for consequential decisions.