groundedness-judge-lfm2.5

Groundedness Judge — LFM2.5-2.6B (LoRA)

A tiny, fully local groundedness auditor for AI agents. Given (1) the context an agent had available, (2) the user's question, and (3) the agent's response, it decides whether the response is grounded in that context and returns a structured JSON verdict — flagging invented facts, missing citations, and avoidable gaps.

Built as a LoRA adapter on LiquidAI/LFM2.5-2.6B, so it runs on modest hardware (trained on a single Tesla P100; small enough to run on a phone at Q4).

Trained by MrWoRmMr.

Why

Local agent stacks hallucinate, and shipping a cloud "judge" defeats the point of running locally. This is a small judge you can run next to your agents, offline, to catch groundedness failures.

Task & output schema

The model returns a single JSON object:

Anti-artifact guards (things that are not hallucinations): addressing the user by name, naming the system/ecosystem, naming a teammate agent, stable general knowledge, and arithmetic correctly derived from given numbers.

Training

• Base: LiquidAI/LFM2.5-2.6B · Method: LoRA (rank 16, α 32, dropout 0.05, all linear modules)
• Data: a small synthetic dataset of audit cases (balanced hallucination / grounded). Synthetic on purpose — reproducible and privacy-preserving; the model never sees real user data.
• Hyperparams: 6 epochs, lr 1.5e-4 (cosine), fp16, cutoff 1024, single Tesla P100 (Pascal).
• Result: train_loss ≈ 0.25 in ~4 min.

Reproduction gotchas (Pascal / LFM2.5)

• LFM2.5's tokenizer needs transformers 5.x; but transformers 5.x's moe.py needs torch ≥ 2.5 (older torch fails infer_schema on string annotations). On a CUDA-12.2 driver, torch 2.5.1+cu121 is the sweet spot.
• Pascal (sm_60) has no bf16 and no FlashAttention → train with bf16: false, fp16: true, flash_attn: disabled. Plain fp16 LoRA fits a 2.6B in 16 GB; no 4-bit needed.

Known limitation (read this)

The model reliably reaches the correct judgment, but on out-of-distribution inputs it tends to reason in prose instead of emitting clean JSON. Force structured output at inference:

• llama.cpp / Ollama: use a JSON grammar (GBNF) or format: json
• transformers: constrained decoding / a JSON schema

With constrained decoding you get valid JSON every time; the fine-tune supplies the judgment, the grammar supplies the format.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

tok = AutoTokenizer.from_pretrained("MrWoRmMrLabs/groundedness-judge-lfm2.5")
base = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM2.5-2.6B")
model = PeftModel.from_pretrained(base, "MrWoRmMrLabs/groundedness-judge-lfm2.5")

# messages = [{"role":"system","content": RUBRIC}, {"role":"user","content": AUDIT_CASE}]

(Prefer constrained JSON decoding — see Known limitation.)

License

LoRA weights under the LFM Open License v1.0 (inherited from the base model): free for organizations under $10M annual revenue; commercial licensing above that threshold.