Validation of a hospital-wide risk stratification model for predicting code events among critically ill children.

OBJECTIVE

The pediatric Critical event Risk Evaluation and Scoring Tool (pCREST) is a validated, machine learning model that continuously predicts the risk of deterioration in children across hospital units. Our objective was to test the performance of pCREST in the early identification of code events in the pediatric intensive care unit (PICU).

METHODS

This was a single-center, retrospective external validation study (2020-2023). The primary outcome was a code (receipt of cardiopulmonary resuscitation and/or code-dose epinephrine) within 12 h of any recorded vital sign or laboratory result. Code events were extracted from our hospital communications platform and electronic health record, and the first code event for each patient was identified. pCREST scores were calculated for each observation and their predictive performance assessed across various time points. Transfer learning was performed via a recurrent neural network (pCREST-RNN).

RESULTS

Of 5254 encounters in the cohort, 2.6% experienced at least one code event. pCREST demonstrated good performance in predicting code events within 12 h (c-statistic 0.80), and also demonstrated a 13-fold improvement in detecting the outcome over the outcome's prevalence (area under the precision recall curve: 0.034). Sensitivity analysis demonstrated high performance up to 18 h before the event. Transfer learning did not improve performance.

CONCLUSIONS

pCREST performs well at predicting code events in external setting at several time points before the outcome.