cranebear-extraction-4b-gguf

CraneBear Extraction 4B (GGUF)

A QLoRA fine-tune of Qwen3-4B that turns a panic emergency voice transcript into structured extraction JSON for a human dispatcher. Quantized to Q4_K_M (~2.3 GB) for on-device / edge use.

The model extracts. A human decides. It never triages, dispatches, or advises. Severity bands are evidence labels for a trained dispatcher, not autonomous decisions.

This checkpoint is round 3 of the CraneBear extraction fine-tune — the round that clears the partner's blocking safety gate when served correctly.

---

⚠️ Read this first: presence schema + exact system prompt

Two serving mistakes silently destroy accuracy (one of them looked like a 41-point regression during development):

1. Use the shipped SYSTEM prompt byte-for-byte. Do not substitute a different extractor prompt, and do not use an older Modelfile from a prior round. The weights and the prompt are a matched set.
2. Emit the presence schema, not the closed schema. The model always outputs explicit sentinels (NOT_REPORTED, NO_EMERGENCY, NOT_APPLICABLE) and presence booleans — never null. Map to nulls in application code (or use the partner's GeneratedMapping). Pairing these weights with a grammar that allows null recreates the drift bug.

Ship with this repo:

---

Output schema (presence)

{
  "casualty_count_stated": true,
  "casualty_count": 1,
  "severity": "IMMEDIATE",
  "severity_confidence": "HIGH",
  "hazards_assessed": true,
  "hazards": {
    "fire": false,
    "gas_chem": false,
    "electrical": false,
    "bio": false
  },
  "summary": "Caller reports trapped person, heavy bleeding."
}

Closed-schema consumers map sentinels → null (severity/confidence/hazards) and casualty_count_stated=false → casualty_count=null.

---

Measured quality (held-out seed.jsonl, 24 cases)

With the shipped prompt + presence constraint + deterministic post-guards:

The ~2-point Ollama vs llama.cpp gap is two near-tie presence-boolean flips on a 24-case set (one field flip ≈ 1.04 pts of macro). Both runs pass the blocking gate.

Guardrails (confidence clamps for hedged/degraded transcripts, raise-only severity floor, summary length/GSM-7/non-verbatim repairs) are part of the published score. Without them, macro is still above 90% with 0% missed-IMMEDIATE; confidence accuracy is lower because teacher labels disagreed with the partner's HIGH rubric on hedged cases.

---

Usage

Ollama

hf download ajvikram/cranebear-extraction-4b-gguf cranebear-extraction-4b-q4_k_m.gguf
hf download ajvikram/cranebear-extraction-4b-gguf Modelfile

ollama create cranebear-extraction -f Modelfile

curl -s http://localhost:11434/api/generate -d '{
  "model": "cranebear-extraction",
  "prompt": "Transcript:\nHe is under the car, not responding, a lot of blood on the pavement.",
  "format": "json",
  "stream": false,
  "options": { "temperature": 0, "num_ctx": 2048 }
}'

Prompt the user turn as Transcript:\n... — that is how the model was trained.

llama.cpp (on-device)

Use cranebear-extraction-presence.gbnf with root symbol root, greedy decoding (temperature 0), context 2048, and the same chat template as the Modelfile (Qwen3 <|im_start|> with the thinking block closed empty).

# Example shape — paths/flags vary by build
./llama-cli -m cranebear-extraction-4b-q4_k_m.gguf \
  --grammar-file cranebear-extraction-presence.gbnf \
  -n 320 -c 2048 --temp 0

---

Training summary

Full writeup of the five rounds, prompt-drift incident, and guardrails: SLM_TRAINING.md (if mirrored) / project docs in the training repo.

---

Intended use / limitations

• Intended: on-device / edge extraction of structured fields from short emergency voice transcripts for a human dispatcher.
• Not intended: autonomous triage, medical advice, or replacing a dispatcher.
• Eval set is small (24 cases). Treat point estimates as directional; the blocking under-triage metric is the load-bearing claim.
• Synthetic training data from a teacher model; partner gold is eval-only.

---

License

Apache-2.0 (inherits from Qwen3-4B).