sauerkraut-gliner-lfm-GGUF

SauerkrautLM-LFM2.5-GLiNER GGUF

GGUF conversions of VAGOsolutions/SauerkrautLM-LFM2.5-GLiNER for CrispEmbed inference.

Zero-shot Named Entity Recognition — detect arbitrary entity types at inference time, no retraining needed.

Model variants

All variants produce the same entities. Score deltas vs F32: Q8_0 ≤ 0.01, Q4_K ≤ 0.03.

Architecture

LFM2.5-350M bidirectional backbone (16 layers: 10 ShortConv + 6 GQA attention, SwiGLU FFN) + layer fusion (squeeze-and-excitation) + BiLSTM + GLiNER span-label matching head.

Usage

# CLI
./crispembed -m gliner-lfm-q8_0.gguf \
  --ner "Tim Cook announced the new iPhone in Cupertino" \
  --ner-labels "person,organization,location,product" --json

# Server
./crispembed-server --ner gliner-lfm-q8_0.gguf --port 8080
curl -X POST http://localhost:8080/ner/extract \
  -d '{"text": "Tim Cook at Apple", "labels": ["person", "organization"]}'

from crispembed import CrispNER

ner = CrispNER("gliner-lfm-q8_0.gguf")
entities = ner.extract(
    "Maria Schmidt arbeitet bei Siemens in München",
    labels=["person", "organization", "location"],
)
for e in entities:
    print(f"{e['text']} => {e['label']} ({e['score']:.2f})")

Parity

All 16 backbone layers cos=1.000000 vs HuggingFace Python reference. 17/17 entities match across 5 test texts.

License

LFM Open License v1.0 — free for entities under $10M annual revenue. See upstream license.

Conversion

python models/convert-gliner-lfm-to-gguf.py \
  --model VAGOsolutions/SauerkrautLM-LFM2.5-GLiNER \
  --output gliner-lfm-f32.gguf
./crispembed-quantize gliner-lfm-f32.gguf gliner-lfm-q8_0.gguf q8_0

Provenance and EU AI Act Art. 53 note

• Upstream model: VAGOsolutions/SauerkrautLM-LFM2.5-GLiNER — published by VAGOsolutions.
• Upstream licence: other. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.