Development and external validation of a machine learning model for predicting chronic critical illness in ICU patients with acute pancreatitis.

BACKGROUND

To develop and validate a machine learning (ML) model to assess the risk of chronic critical illness (CCI) in intensive care unit (ICU) patients with acute pancreatitis (AP).

METHODS

We utilised two large, publicly available ICU datasets, MIMIC-IV (v3.1) and the eICU Collaborative Research Database (v2.0), as the development cohort for model construction. A single-centre dataset from China (SZICU) was used for external validation. Three feature selection methods-stepwise regression, Least Absolute Shrinkage and Selection Operator (LASSO), and the Boruta algorithm-were employed. Three ML methods-logistic regression (LR), random forest (RF), and extreme gradient boosting (XGBoost)-were used for model development. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, F1 score, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Brier score, in both internal and external validation.

RESULTS

The incidences of CCI were 7.00%, 9.89%, and 20.09% in the training, internal validation, and external validation sets, respectively. Eight predictors of CCI were identified: calcium level, body temperature, vasopressor use, urine output, Glasgow Coma Scale score, albumin level, haemoglobin level, and a history of cerebrovascular disease. In the internal validation set, the RF model achieved an AUROC of 0.85 (0.77-0.91), an AUPRC of 0.53 (0.39-0.69), and a Brier score of 0.07 (0.05-0.09). In the external validation set, the RF model achieved an AUROC of 0.73 (0.64-0.81), an AUPRC of 0.42 (0.30-0.56), and a Brier score of 0.16 (0.12-0.20). Feature importance analysis revealed that calcium level, body temperature, vasopressor use, and urine output were the most influential predictors of CCI.

CONCLUSIONS

We developed and validated an ML model using eight clinical variables to predict CCI risk in ICU patients with AP.