Multistep validation of a post-ERCP pancreatitis prediction system integrating multimodal data: a multicenter study.

BACKGROUND AND AIMS: The impact of various categories of information on the prediction of post-ERCP pancreatitis (PEP) remains uncertain. We comprehensively investigated the risk factors associated with PEP by constructing and validating a model incorporating multimodal data through multiple steps.

METHODS

Cases (n = 1916) of ERCP were retrospectively collected from multiple centers for model construction. Through literature research, 49 electronic health record (EHR) features and 1 image feature related to PEP were identified. The EHR features were categorized into baseline, diagnosis, technique, and prevention strategies, covering pre-ERCP, intra-ERCP, and peri-ERCP phases. We first incrementally constructed models 1 to 4 incorporating these 4 feature categories and then added the image feature into models 1 to 4 and developed models 5 to 8. All models underwent testing and comparison using both internal and external test sets. Once the optimal model was selected, we conducted comparisons among multiple machine learning algorithms.

RESULTS

Compared with model 2 that incorporated baseline and diagnosis features, adding technique and prevention strategies (model 4) greatly improved the sensitivity (63.89% vs 83.33%, P < .05) and specificity (75.00% vs 85.92%, P < .001). A similar tendency was observed in the internal and external tests. In model 4, the top 3 features ranked by weight were previous pancreatitis, nonsteroidal anti-inflammatory drug use, and difficult cannulation. The image-based feature has the highest weight in models 5 to 8. Finally, model 8 used a random forest algorithm and showed the best performance.

CONCLUSIONS

We first developed a multimodal prediction model for identifying PEP with a clinical-acceptable performance. The image and technique features are crucial for PEP prediction.