How Can Clinicians Decide if AI-Enabled Risk Prediction Models Are Fit for Purpose for Real World Use?

Artificial intelligence (AI)-enabled risk prediction models are being increasingly used in hospital practice to predict patient risk of adverse events, aimed at giving early warning of impending events and facilitating preventive intervention. However, evidence of beneficial impact varies which may be due to clinicians not finding them useful because of suboptimal predictive accuracy (effectiveness), burden of false alerts (efficiency) or narrow prediction windows (utility). Current model performance measures do not capture the interdependency of these three dimensions. In this commentary, using sepsis risk prediction as an example, we present methods that assist clinicians to: assess the suitability of particular models and decide ideal decision thresholds; optimise model performance; and configure and deliver alerts to frontline clinicians.