Rethinking Diagnostic Performance Metrics in Imaging Artificial Intelligence: Lessons from Caries Detection on Radiographs.

BACKGROUND

Artificial intelligence (AI) systems for radiographic caries detection are commonly evaluated using a small set of performance metrics, yet these measures are frequently reported and interpreted without the contextual information required for clinically meaningful conclusions.

SUMMARY: This narrative review focuses on performance metrics - confusion-matrix measures (sensitivity, specificity, accuracy, predictive values, F1 score and related summaries) and discrimination metrics (area under the receiver operating characteristic curve and area under the precision recall curve) - and discusses how their interpretation depends on explicit reporting of the disease definition (grading cut-offs), unit of analysis (tooth surface/patient), decision threshold(s), and prevalence/case-mix. We additionally cover complementary metric-based assessments, including calibration (calibration curves, expected calibration error, Brier score), decision-analytic metrics (decision curve analysis and cost-sensitive clinical loss), and robustness summaries (uncertainty and subgroup/worst-group performance). A structured summary is provided of formulae, interpretive limitations, and the minimum information that should accompany each metric to support transparent reporting, comparability, and transportability.

KEY MESSAGES: AI-assisted radiographic caries detection cannot be judged by headline metrics alone; traditional performance measures should be reported with clear disease definitions, operating points, prevalence, and the clinical impact of false results. We provide a practical guide to improving the transparency, comparability and clinically meaningful interpretation of future studies.