External Validation of Two AI Systems for Dentinal Caries Detection: A Retrospective Pilot Clinical Outcomes Study.

OBJECTIVES

To evaluate the sensitivity of two dental artificial intelligence (AI) systems for detecting dentinal caries using longitudinal, outcomes-anchored validation within an integrated healthcare system.

METHODS

In this pilot external validation study, two commercially available AI systems were tested on full-mouth intraoral radiographic series, including bitewings, from 90 patients in a federal healthcare system. Ground truth was established using baseline clinical examinations and follow-up documentation 6-12 months later. Vendor-specific and pooled sensitivity for dentinal caries detection were calculated.

RESULTS

Vendor A achieved a sensitivity of 48.5% (95% CI: 37.0-60.2), Vendor B achieved 57.5% (95% CI: 46.1-68.2), and pooled sensitivity was 53.0% (95% CI: 44.8-61.1).

CONCLUSIONS

Outcomes-anchored, longitudinal validation of dental AI systems for dentinal caries detection was feasible within a large integrated healthcare system. This approach provides clinically relevant sensitivity estimates grounded in real treatment outcomes. These findings are most directly applicable to large integrated healthcare systems with standardized longitudinal data capture and may not fully reflect the variability of private-practice dental settings.

ADVANCES IN KNOWLEDGE: This study establishes a reproducible framework for evaluating AI diagnostic performance in real-world healthcare settings using longitudinal clinical outcomes rather than radiographic labels alone, demonstrating that independent validation with clinical follow-up is feasible within a large healthcare system.