prompt-surgeon-lfm2.5

Prompt Surgeon — LFM2.5-2.6B (LoRA)

A tiny, fully local prompt surgeon for AI agents. Given an agent's current prompt and evidence of a failure (a hallucination case plus a groundedness auditor's verdict), it proposes a single surgical fix — one rule to ADD to the prompt that prevents that whole class of failure — or says no change needed when the response was actually fine.

It is the companion to the groundedness-judge: the judge detects the hallucination, this model prescribes the cure. Together they close a measure → fix → re-measure loop that a local agent stack can run on itself, offline.

Built as a LoRA adapter on LiquidAI/LFM2.5-2.6B — small enough to run on a phone at Q4, trained on a single Tesla P100.

Trained by MrWoRmMr.

Task & output schema

Input: the agent's current prompt (excerpt) + the failure evidence (bad response + auditor verdict). Output: a JSON object with

It is surgical, not a rewrite: it adds one targeted rule instead of rewriting the whole prompt — safer (won't break the persona), composable, and learnable. It also learns restraint: on grounded responses (or anti-artifact cases like naming the operator), it returns target: "none" and adds nothing.

Training

• Base: LiquidAI/LFM2.5-2.6B · Method: LoRA (rank 16, α 32, dropout 0.05, all linear modules)
• Data: a small synthetic dataset of (prompt + failure → surgical fix) cases, balanced between "fix" and "no-change" so the model learns when not to prescribe. Synthetic on purpose — reproducible and privacy-preserving; it never sees real user data.
• Hyperparams: 6 epochs, lr 1.5e-4 (cosine), fp16, cutoff 1024, single Tesla P100 (Pascal).
• Held-out check: correctly diagnoses failures and abstains on grounded cases.

Reproduction gotchas (Pascal / LFM2.5)

• LFM2.5's tokenizer needs transformers 5.x; but transformers 5.x needs torch ≥ 2.5. On a CUDA-12.2 driver, torch 2.5.1+cu121 is the sweet spot.
• Pascal (sm_60) has no bf16 and no FlashAttention → bf16: false, fp16: true, flash_attn: disabled.

Known limitation (read this)

Like its judge sibling, the model reliably reaches the correct fix, but on out-of-distribution inputs it tends to reason in prose before emitting JSON. Force structured output at inference (llama.cpp / Ollama JSON grammar or format: json; transformers constrained decoding). The fine-tune supplies the judgment, the grammar supplies the format.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

tok = AutoTokenizer.from_pretrained("MrWoRmMrLabs/prompt-surgeon-lfm2.5")
base = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM2.5-2.6B")
model = PeftModel.from_pretrained(base, "MrWoRmMrLabs/prompt-surgeon-lfm2.5")
# feed: system rubric + (agent prompt + failure evidence). Prefer constrained JSON decoding.

License

LoRA weights under the LFM Open License v1.0 (inherited from the base model): free for organizations under $10M annual revenue; commercial licensing above that threshold.