A Comparative Analysis of Machine Learning Models for Early Detection of Hospital-Acquired Infections

As more and more infection-specific machine learning models are developed and planned for clinical deployment, simultaneously running predictions from different models may provide overlapping or even conflicting information. It is important to understand the concordance and behavior of parallel models in deployment. In this study, we focus on two models for the early detection of hospital-acquired infections (HAIs): 1) the Infection Risk Index (IRI) and 2) the Ventilator-Associated Pneumonia (VAP) prediction model. The IRI model was built to predict all HAIs, whereas the VAP model identifies patients at risk of developing ventilator-associated pneumonia. These models could make important improvements in patient outcomes and hospital management of infections through early detection of infections and in turn, enable early interventions. The two models vary in terms of infection label definition, cohort selection, and prediction schema. In this work, we present a comparative analysis between the two models to characterize concordances and confusions in predicting HAIs by these models. The learnings from this study will provide important findings for how to deploy multiple concurrent disease-specific models in the future.

Predicting Hyperkalemia in the ICU and Evaluation of Generalizability and Interpretability

Hyperkalemia is a potentially life-threatening condition that can lead to fatal arrhythmias. Early identification of high risk patients can inform clinical care to mitigate the risk. While hyperkalemia is often a complication of acute kidney injury (AKI), it also occurs in the absence of AKI. We developed predictive models to identify intensive care unit (ICU) patients at risk of developing hyperkalemia by using the Medical Information Mart for Intensive Care (MIMIC) and the eICU Collaborative Research Database (eICU-CRD). Our methodology focused on building multiple models, optimizing for interpretability through model selection, and simulating various clinical scenarios. In order to determine if our models perform accurately on patients with and without AKI, we evaluated the following clinical cases: (i) predicting hyperkalemia after AKI within 14 days of ICU admission, (ii) predicting hyperkalemia within 14 days of ICU admission regardless of AKI status, and compared different lead times for (i) and (ii). Both clinical scenarios were modeled using logistic regression (LR), random forest (RF), and XGBoost. Using observations from the first day in the ICU, our models were able to predict hyperkalemia with an AUC of (i) 0.79, 0.81, 0.81 and (ii) 0.81, 0.85, 0.85 for LR, RF, and XGBoost respectively. We found that 4 out of the top 5 features were consistent across the models. AKI stage was significant in the models that included all patients with or without AKI, but not in the models which only included patients with AKI. This suggests that while AKI is important for hyperkalemia, the specific stage of AKI may not be as important. Our findings require further investigation and confirmation.