oac-system-health-v1

OAC System Health v1

OAC System Health v1 is a tiny, dependency-free logistic-regression model for synthetic operational transport telemetry. It emits a non-authoritative operator-attention advisory. The artifact files reproduce byte-for-byte from szl-forge commit 0e9812f989b3548ddcff7e24443da3cb5764cf50 (see Provenance).

Companion data: synthetic transport observability dataset, revision f7ab6170. Source at the pinned commit: szl-forge clinical-gateway/ at 0e9812f9. That directory also hosts a separate integration stack. This model consists only of src/oac_operational_health.py, tools/train_operational_health_model.py, operational-model/ and huggingface/ in that directory, and it is bounded by the section below.

Evidence labels used on this card: MEASURED (re-run and checked by SZL Holdings on 2026-09-27 to 2026-09-29 against the files at the revisions named here), REPORTED (taken from the named source), MODELED (follows from reading the code), UNKNOWN (not established). SYNTHETIC marks every number computed on generated data.

Critical boundary

This is not a medical, diagnostic, prognostic, triage, treatment, or clinical decision model. It was not trained or validated on patients, laboratory results, specimens, orders, assays, physical devices, or a health-care site. It must never receive PHI or clinical/result content.

The model cannot:

• accept, reject, or acknowledge an HL7 message;
• command or identify a medical device;
• interpret, validate, autoverify, route, or release a result;
• authorize clinical use; or
• establish regulatory, privacy, security, or site acceptance.

<!-- SZL-NATIVE-PRESENTATION:v1 --> <p><a href="https://huggingface.co/SZLHOLDINGS"><img src="https://raw.githubusercontent.com/szl-holdings/szl-brand/27d12e77fc0f9f1a0e5fae8b34adf2380d51d070/kit/logos/orbit-v2/suite/szl-icon-32.svg" alt="SZL Holdings" width="48" height="48" /></a></p>

Command Lab · Canonical source · Evidence

<!-- /SZL-NATIVE-PRESENTATION:v1 -->

Provenance

• Artifact revision. model.json, artifact_receipt.json, oac_operational_health.py, example_input.json and LICENSE are the files published at Hub revision dd7d109813abcd90c5250106dc2eabd2804e7ac3 (2026-09-24). This card revision changes README.md only. The kernel never reads README.md and no receipt hashes it, so model_sha256 and every other receipt hash are unchanged. [MEASURED for the file hashes; MODELED from the kernel and trainer code for the README independence]
• Source commit. szl-forge 0e9812f989b3548ddcff7e24443da3cb5764cf50 (merge of pull request #351, 2026-09-24). How it was verified:
  1. The Hub commit dd7d1098 names this commit in its message. [REPORTED: Hub commit history of this repository]
  2. Each of the six model files at Hub revision dd7d1098 has the same SHA-256 as its copy under clinical-gateway/huggingface/model/oac-system-health-v1/ at this commit. The eight dataset files at dataset revision f7ab6170 also match, and the four hashes in artifact_receipt.json reproduce. [MEASURED]
  3. szl-forge's own tools/verify_hub_alignment.py (verify_alignment()), run offline against the downloaded Hub files for this commit, reports complete: true. [MEASURED]
  4. tools/train_operational_health_model.py --verify rebuilds the data, model and receipts from seed 2500 in a temporary directory and byte-compares them with the 22 committed paths: no mismatch. [MEASURED at 0e9812f9 and at its descendant 9a6fdd2c]
• What the pin does not prove. The pin is unsigned (source_signature_verified: false): no signature binds these files to the commit. Every szl-forge commit on main from eb871c9b (2026-09-08) through 9a6fdd2c reproduces the same bytes, so which commit actually ran the trainer is UNKNOWN. [MEASURED; UNKNOWN as stated]
• This card's own text comes from a later szl-forge commit, which the Hub commit that publishes this card names in its message. A file cannot contain the hash of the commit that contains it.

Inputs

The kernel requires exactly eight operational fields:

Binary fields are used as is. The three continuous fields are min-max scaled with the bounds above (feature_manifest in model.json).

When every JSON key appears once, unknown, missing, non-finite, out-of-range, identity-like, HL7, FHIR, patient, specimen, order, and result fields fail closed. [MEASURED] Known gap: the v1 kernel parses duplicate JSON keys last-wins, so an earlier duplicate (for example a first features object that carries a prohibited field name) is dropped before these checks run, and the input is accepted. Reject duplicate keys before calling the kernel until a kernel fix ships. [MEASURED]

Run without third-party packages

python -I -B oac_operational_health.py \
  --model model.json \
  --receipt artifact_receipt.json \
  --input example_input.json

The kernel verifies the model SHA-256 from artifact_receipt.json before inference (model_sha256 f111b7fc65db561c80763b25e538366a19bed18526ca881915777487935d9557). Output includes operator_attention_score, the validation-selected threshold, the boolean advisory, per-feature contributions, and an explicit all-false authority map.

The receipt is a reproducibility/mismatch control, not a signature or external trust root. A trusted deployment must pin the Hub commit and verify it through its own software-supply-chain policy; replacing both the model and receipt can otherwise bypass this local comparison.

Training and evaluation

The model is batch-gradient-descent logistic regression implemented using only the Python standard library (2400 epochs, learning rate 0.42, L2 penalty 0.01). Fixed seed 2500 generates 768 training, 192 validation, and 240 test examples. The receipt records exact split metrics and hashes. Validation chooses the decision threshold by balanced accuracy, then F1; the chosen threshold is 0.16. The test split was not used for fitting or for threshold selection. It is a public split that was already used during development, so it is not a new blind evaluation.

Synthetic labels are sampled from a logistic function of the same eight normalized inputs (_synthetic_label_probability in the dataset's training_source_snapshot.py). The model family therefore matches the generator by construction, and the metrics measure how well v1 recovers that generator under label noise, not signal in real telemetry. [MODELED]

All reported metrics are from generated synthetic examples. The score is not production-calibrated and the metrics must not be generalized to a real transport, analyzer, laboratory, patient population, or clinical workflow.

Synthetic test results, with intervals and baselines

Every number in this section is SYNTHETIC. Real-world performance, calibration and alert burden are UNKNOWN.

The test split has 240 rows, 51 of them positive. At threshold 0.16 the confusion counts are 40 true positives, 57 false positives, 132 true negatives and 11 false negatives. Rescoring all 240 rows with this kernel reproduces the receipt's counts exactly. [MEASURED, SYNTHETIC]

Accuracy is not used for selection: with 51 positives in 240 test rows, the majority baseline is correct on 189 rows (0.7875) without raising a single alert, while v1 is correct on 172 (0.7167). [MEASURED, SYNTHETIC]

Each cell is the point estimate followed by a 95% percentile bootstrap interval: 2,000 paired resamples of the 240 test rows, drawn with Python random.Random(20260926), percentiles by linear interpolation. All three classifiers are scored on the same resamples. [MEASURED, SYNTHETIC]

• Majority baseline: always predicts the training majority class (no attention) and scores every row with the training prevalence 151/768. It never alerts, so its precision and false-alert share are undefined.
• Rule baseline: alerts when consecutive_failures > 0 and scores the row 0 or 1. For a 0/1 score, the Brier score and ECE both equal the error rate.

Paired differences on the same resamples, v1 minus baseline:

What this shows, on this synthetic split only:

• Most alerts are false. The false-alert share is 0.5876 [0.4896, 0.6869].
• v1 beats both baselines on balanced accuracy, ROC AUC and Brier score; those paired intervals exclude 0.
• v1's specificity is not distinguishable from the rule's.

Known limitations

• Duplicate JSON keys bypass the fail-closed checks (see Inputs). [MEASURED]
• At inference the kernel checks only model_sha256. The receipt's kernel_sha256, generator_sha256 and dataset_receipt_sha256 are not checked. [MEASURED]
• A few extreme inputs (very large integer literals, very deeply nested arrays) are refused through an uncaught exception (exit 1 with a traceback) instead of the documented exit-2 JSON error. [MEASURED]
• The receipt is not a signature (see Run without third-party packages).
• The generator and the model share a functional form (see Training and evaluation). No evidence beyond synthetic data exists. [MODELED; UNKNOWN for real telemetry]

REFORMS checklist

The 32 items of the REFORMS reporting checklist for ML-based science (arXiv:2308.07832v2), with this card's status. Source id is the checklist's own item id. The item text is SZL's paraphrase, not a quotation. Totals: 12 satisfied, 12 partial, 4 not applicable, 4 missing.

License

Apache-2.0. See LICENSE.

<!-- SZL-NATIVE-ARTWORK:v1 -->

SZL Orbit artwork: source · CC BY 4.0. Software and artifact licensing remain declared above.

<!-- /SZL-NATIVE-ARTWORK:v1 -->