bert-esi-triage-v58

BERT-ESI-Triage v58 — BiomedBERT multi-head triage classifier

TL;DR

bert-esi-triage-v58 is a fine-tuned BiomedBERT classifier for the Emergency Severity Index (ESI 1-5) triage decision. v58 is the successor to v57 (epoch 3) on the same 18-eval suite. Headline improvements vs v57 e3:

• MIMIC-IV-ED ESI 1 recall: +4.0pp ensemble (78.1% → 82.1%).
• MC-MED Stanford ESI 1 recall: +7.0pp (77.0% → 84.0%).
• MIETIC ensemble ESI 1 recall held at 96.7%.
• Lukina v3 ESI 1 recall held at 94.3%.
• Concept-density / frequent-flyer / polypharmacy / geriatric / multi-CC subgroup slices all win ESI 1 recall by +1.6 to +6.7pp.

Trade-off: cal_exact regresses by 2-6pp on MIETIC, judgment_gap, cardiac_arrest, and sepsis slices (a recall-for-safety shift).

Architecture differences vs v57

• No engineered-feature fusion. v57 trained with an 11-d MEWS / qSOFA / shock_index feature vector projected to 64-d and concatenated with CLS into esi_head Linear(832, 5). v58 drops the fusion path and uses esi_head Linear(768, 5) directly on CLS. The encoder learns vital patterns from raw text alone.
• Trainer. v58 uses train_bert_v44_bf16.py (bf16 native on GB10); v57 used train_bert_v44_v53_3.py (feature-fusion variant).
• Symptom head dim. v58 = 199, v57 = 203 (concept vocab pruning).

Inference

Load with PyTorch + transformers; CLS-pooled features feed each head:

• esi_head: Linear(768, 5) — main output.
• 19 auxiliary heads: symptom, flag, pain, arrival, gestalt, disposition, resource, vitals, etc.

The companion Space vadimbelsky/esi-triage-demo ships the full calibration stack (demographic normalizer → per-dialect temperature → confidence-aware ESI 1 bias → deterministic engine min ensemble).

License

• Model weights: MIT.
• Training data: private (MIMIC-IV-ED + Stanford MC-MED + MIETIC + Lukina v3 + curated synth).

⚠️ Not a medical device. Not for clinical use. This is a research artifact distributed for reproducibility and benchmarking. Real deployment requires institutional validation + IRB + local recalibration.

Performance — v58 epoch 3 vs v57 epoch 3

Per-slice ESI 1 recall (calibrated):

Verdict: 8 wins / 5 losses / 5 neutral on ESI 1 recall (mean +0.78pp).

MIMIC per-class recall (calibrated):

Known gaps (carried to v59)

• SYMPTOM_LABELS index mismatch. v58 symptom_head outputs 199 logits; the demo's SYMPTOM_LABELS list (inherited from v47/v49) has 178 canonical entries plus runtime padding for the extra positions. Specific concepts that grew between v57→v58 may be mislabeled in the Space UI. Tracked for v59 (task #463).
• ESI 5 recall ≤ 40% across slices — sparse class, dialect-noisy labels. v58 holds the v57 floor but does not improve it.
• MIETIC / OB / cardiac_arrest ESI 1 R regressions vs v57. Likely driven by dropping the engineered-feature signal on narrative slices where vitals are inline rather than parsed.
• Engineered features no longer in graph. Downstream stacks that computed MEWS / qSOFA / shock_index and passed them to v57 must drop that path for v58.

Companion Space

vadimbelsky/esi-triage-demo — 19-head dashboard with engine ensemble, demographic normalizer, dialect detection, per-dialect calibration. The Space loads this model and exposes the full calibrated pipeline.