rural-hospital-closure-risk

Model Card: Rural Hospital Closure-Risk Indicator (RHCRI)

Summary

RHCRI estimates the financial vulnerability and modeled closure risk of rural U.S. hospitals from public data. It is a hybrid: a transparent 0–100 Financial Vulnerability Score plus a gradient-boosted closure-risk classifier trained on real historical closures (see the honesty note in Evaluation below — trust the vulnerability score more than the raw ML probability right now). Output is a probability, a risk tier (Low / Moderate / High / Severe, recalibrated for the real base rate), and the top contributing factors.

• Developed by: Ravindra Kumar
• Model type: HistGradientBoostingClassifier + transparent rule-based score
• License: Apache-2.0
• Serialization: skops (artifacts/closure_model.skops) — deliberately not joblib/pickle, which executes arbitrary code on load.
• Intended use: Community, policymaker, and researcher decision support.

Intended use & users

• Intended: early identification of at-risk rural hospitals for advocacy, policy targeting, and resource planning; research on rural access.
• Users: rural-health researchers, community leaders, health departments, journalists, policymakers.
• Out of scope: investment/credit decisions; any claim that a named hospital will definitely close; individual clinical or employment decisions.

Inputs / outputs

• Input: operating & total margin, days cash on hand, occupancy, public-payer share, beds, Critical-Access status, ownership, county poverty, population change, distance to next hospital.
• Output: closure probability (horizon in config), risk tier, vulnerability score, and the specific distress signals driving it.

Training data

• Labels: UNC Sheps Center rural hospital closure list (public xlsx, 204 closures 2005-2026), matched to CMS records by normalized hospital name + state (no CCN in the Sheps list).
• Features: CMS's pre-summarized Hospital Provider Cost Report (data.cms.gov, not the raw HCRIS worksheets) for financials/ownership/ CAH flag/beds; Census ACS for county poverty + population change; the Census Batch Geocoder for straight-line distance to the nearest other hospital. All public, hospital/county level. No patient data.
• ⚠ Critical scope limit, found while building this: the CMS cost-report extract is a rolling "latest filed report" snapshot (FY2022-2024 at time of writing), not a historical archive. A hospital's financials must predate its closure for a valid label, so only Sheps closures from 2022 onward are usable here — of those, only 12 could be matched by name to a CCN in the cost-report data (name-matching health-system rebrand names like "Northern Light Inland Hospital" to legal CCN names like "Inland Hospital" is imperfect; exact-match only was used deliberately, to avoid a fuzzy-match false-labeling a still-open hospital as closed). Final training set: 12 real confirmed closures out of 2,671 real rural hospitals (~0.4% base rate). Missing feature values (ungeocodable addresses, unmatched counties — up to ~17% null on some columns) were median-imputed rather than dropped, specifically to avoid discarding already-scarce positive examples; this is a deliberate, documented choice given the label scarcity, not a default practice.

Evaluation — read this before trusting the probability

• Held-out (stratified 80/20) metrics on the 2,671-row real dataset: ROC-AUC 0.90, PR-AUC 0.26, Brier 0.005, but the test split contained only 2 positive examples — these numbers have almost no statistical power and should not be quoted as if they were precise.
• A concrete, checked finding: predicted probabilities on hospitals in the training set that are true closures average ~0.9997, while a hand-constructed "distressed hospital" profile not in the training set scored a mere ~0.0000043 — actually lower than a hand-constructed "healthy" profile. This points to the classifier memorizing the 12 known closures rather than learning a generalizable distress pattern, an expected failure mode of tree ensembles trained on an extremely rare positive class this small. Practical consequence: don't read the raw probability as a calibrated risk estimate for a hospital that isn't closely similar to one of the 12 known closures. Risk tier cutoffs in config/risk_config.yaml were recalibrated to the model's own real percentile distribution (comments in that file explain why), which keeps the tiers informative even though the underlying probabilities aren't well-calibrated in an absolute sense.
• The transparent Financial Vulnerability Score is the more dependable signal right now — it's a direct, auditable function of the inputs with no small-sample generalization risk. This matches the project's own original design rationale (two signals, one always trustworthy) better than either layer alone.
• Proper out-of-time validation (train on older report years, test on newer closures) needs the full historical HCRIS archive, not this snapshot — tracked as future work.

Limitations & biases

• Closure is rare and multi-causal; financial data lags and misses local politics, mergers, and state policy. Treat outputs as probabilistic signals, not verdicts — and see the Evaluation section above for why the ML signal specifically needs extra skepticism right now.
• Label completeness depends on (a) the closure list itself and (b) exact name+state matching to CMS records, which is conservative by design and therefore under-counts real closures (of 29 Sheps closures from 2021+, only 14 matched by exact name to any CMS record at all, and 12 of those had valid pre-closure financials).
• Small-hospital financials are noisy; a few rows had implausible (near-zero) revenue figures, treated as data-entry errors and excluded from margin calculations (see src/data_sources.fetch_cms_cost_reports).

Ethical considerations

Predicting a specific institution's failure can be self-fulfilling and harmful. This model is deliberately framed as a vulnerability indicator with reasons and uncertainty, built to help communities keep hospitals open. It should never be used to stigmatize an institution or make investment decisions.

How to use

import skops.io as sio
from src.vulnerability import HospitalFinancials
from src.pipeline import score_hospital

path = "artifacts/closure_model.skops"
est = sio.load(path, trusted=sio.get_untrusted_types(file=path))
# build feature_row + HospitalFinancials (see demo.py), then:
print(score_hospital(est, feature_row, fin))

Citation

@software{kumar_rhcri_2026,
  author = {Kumar, Ravindra},
  title  = {Rural Hospital Closure-Risk Indicator (RHCRI)},
  year   = {2026},
  url    = {https://huggingface.co/rakumar25/rural-hospital-closure-risk}
}