medicare-inpatient-model

Medicare Inpatient Outcome Prediction Models

Model ID: Medicare LDS 2023 Inpatient Model Bundle Model Types: XGBoost Regressor & Calibrated Multinomial Logistic Regression Dataset: 2023 CMS Limited Data Set (LDS) Target Level: Inpatient Encounter

---

What the Model Predicts

This bundle contains three distinct models that predict key outcomes for a given inpatient hospital stay:

• Length of Stay (Regression): Predicts the total number of days for the inpatient stay.
• Readmission Probability (Binary Classification): Predicts the probability (from 0.0 to 1.0) that the patient will be readmitted to a hospital within 30 days of discharge.
• Discharge Location Probability (Multiclass Classification): Predicts the probability for each possible discharge location (e.g., Home, Skilled Nursing Facility, Hospice).

---

Intended Use

This model bundle is designed to support a variety of clinical and operational workflows:

• Discharge Planning & Care Management: Identify high-risk patients who may need additional support or a specific type of post-acute care to prevent readmission.
• Resource Planning: Forecast bed-days and resource utilization based on predicted length of stay.
• Actuarial Analysis: Inform risk stratification and cost estimation models.
• Benchmarking: Compare observed outcomes against predicted risks for a given patient population.
• Healthcare Research: Analyze drivers of inpatient outcomes.

Note on Prediction Type: The models are trained for concurrent prediction — they use clinical and demographic data available during the inpatient stay to predict outcomes related to that same stay.

---

Model Performance

These metrics reflect performance on a 20% test set held out from the 2023 CMS LDS data. All values were calculated on unseen data and represent model generalization performance.

Model 1: Length of Stay (XGBoost Regressor)

Model 2: Readmission Probability (Calibrated Logistic Regression)

• AUC ROC: Measures the model's ability to distinguish between patients who will and will not be readmitted (higher is better).
• Brier Score: Measures the accuracy of the predicted probabilities (lower is better).

Model 3: Discharge Location (Calibrated Logistic Regression)

• Accuracy: The overall percentage of times the model predicted the correct discharge location.
• Brier Score (Macro Avg): The average Brier Score across all possible discharge location classes (lower is better).

---

Files Included

• inpatient_models_bundle_medicare_lds_2023_fs.pkl — A compressed pickle file containing the trained models, feature lists, and encoders. The filename may vary based on training parameters (e.g., _fs indicates feature selection was used). The bundle includes:
  • los_model (XGBoost)
  • readmission_model (Calibrated Logistic Regression)
  • discharge_model (Calibrated Logistic Regression)
  • feature_columns_* (specific feature lists for each model)
  • le_discharge (label encoder for discharge location)
• Train Tuva Concurrent Inpatient Models.ipynb — The notebook script used for training, feature selection, and evaluation on Snowflake.
• predict_inpatient.py — An example prediction script for running the bundle on new data.
• feature_fill_rate_inpatient.csv — A diagnostic file detailing the prevalence of each feature in the training dataset.
• inpatient_feature_importance.csv — A file containing the calculated importance of each feature for each of the three models.

---

Understanding Model Artifacts

This repository includes two key CSV files that provide insight into the model's training data and internal logic. These are generated by the training notebook, which also populates corresponding tables in Snowflake for easier querying (FEATURE_FREQUENCY_STATS_INPATIENT and MODEL_FEATURE_IMPORTANCE_INPATIENT).

Feature Fill Rates (feature_fill_rate_inpatient.csv)

This file is a diagnostic tool for understanding the input data used to train the models. It helps you check for data drift or data quality issues.

How to Use: Compare the POSITIVE_RATE_PERCENT from this file with the rates from your own prediction input data. Significant discrepancies can point to data pipeline issues and may explain poor model performance.

Feature Importances (inpatient_feature_importance.csv)

This file provides model explainability by showing which features are most influential for each of the three models.

How to Use: Use this file to understand the key drivers behind the model's predictions. For example, you can filter by MODEL_NAME for the readmission model and sort by IMPORTANCE_RANK to see what most influences readmission risk. This is useful for clinical validation and debugging.

---

Quick Start: End-to-End Workflow

This section provides high-level instructions for running a model with the Tuva Project. The workflow involves preparing benchmark data using dbt, running a Python prediction script, and optionally ingesting the results back into dbt for analysis.

1. Configure Your dbt Project

You need to enable the correct variables in your dbt_project.yml file to control the workflow.

A. Enable Benchmark Marts

These two variables control which parts of the Tuva Project are active. They are false by default.

# in dbt_project.yml
vars:
  benchmarks_train: true
  benchmarks_already_created: true

• benchmarks_train: Set to true to build the datasets that the ML models will use for making predictions.
• benchmarks_already_created: Set to true to ingest model predictions back into the project as a new dbt source.

B. (Optional) Set Prediction Source Locations

If you plan to bring predictions back into dbt for analysis, you must define where dbt can find the prediction data.

# in dbt_project.yml
vars:
  predictions_person_year: "{{ source('benchmark_output', 'person_year') }}"
  predictions_inpatient: "{{ source('benchmark_output', 'inpatient') }}"

C. Configure sources.yml

Ensure your sources.yml file includes a definition for the source you referenced above (e.g., benchmark_output) that points to the database and schema where your model's prediction outputs are stored.

---

2. The 3-Step Run Process

This workflow can be managed by any orchestration tool (e.g., Airflow, Prefect, Fabric Notebooks) or run manually from the command line.

Step 1: Generate the Training & Benchmarking Data

Run the Tuva Project with benchmarks_train enabled. This creates the input data required by the ML model.

dbt build --vars '{benchmarks_train: true}'

To run only the benchmark mart:

dbt build --select tag:benchmarks_train --vars '{benchmarks_train: true}'

Step 2: Run the Prediction Python Code

Execute the Python script (predict inpatient.ipynb) to generate predictions. This script will read the data created in Step 1 and write the prediction outputs to a persistent location (e.g., a table in your data warehouse).

We have provided example Snowflake Notebook code within each model's repository that was used in Tuva's environment.

Step 3: (Optional) Bring Predictions back into Tuva Project

To bring the predictions back into the Tuva Project for analysis, run dbt again with benchmarks_already_created enabled. This populates the analytics marts.

dbt build --vars '{benchmarks_already_created: true, benchmarks_train: false}'

To run only the analysis models:

dbt build --select tag:benchmarks_analysis --vars '{benchmarks_already_created: true, benchmarks_train: false}'

---