Commercial artificial intelligence-assisted performance and interpretation time of first-on-call radiology residents using computed tomography pulmonary angiography to detect pulmonary embolism: a multireader, multicenter study.

BACKGROUND

Acute pulmonary embolism (PE) is characterized by sudden onset, high mortality, and a propensity for misdiagnosis. Although artificial intelligence (AI)-based software for PE detection has been shown to accurately identify thrombi on computed tomography pulmonary angiography (CTPA), its impact on the performance of first-on-call radiologists of varying experience levels during initial interpretation remains unclear. This study aimed to examine the performance and time of first-on-call radiology residents' interpretation of CTPA for acute PE with and without AI support.

METHODS

This retrospective study included 196 consecutive emergency CTPA examinations (55 PE-positive and 243 clots) from three centers between June 2023 and August 2023. Six residents (1-6 years' experience) independently interpreted all CTPA scans with and without National Medical Products Administration-approved AI assistance software that provided PE triage, clot detection, and quantification. The reference standard was established by two senior radiologists. Performance metrics (sensitivity, specificity, and Youden index) and interpretation time were compared. AI dependency was assessed by quantifying AI-induced decision changes.

RESULTS

AI support improved patient-level sensitivity (0.83 to 0.96) and Youden index (0.80 to 0.91), with more pronounced gains at the clot level (sensitivity: 0.75 to 0.95; Youden index: 0.56 to 0.81). Junior residents (<5 years) showed greater improvement than did seniors (≥5 years). Mean interpretation time decreased by 10 seconds (18%) per case (P<0.001), with greater reductions observed among junior residents and for non-PE cases. AI significantly reduced the risk of missed clots [odds ratio (OR) 0.11; 95% confidence interval (CI): 0.08-0.15; P<0.001] and increased the incidence of false positives, but not significantly so (OR 1.46; 95% CI: 0.85-2.52; P=0.17). Junior residents exhibited higher AI dependency, yet the overall rates of incorrect AI guidance remained low (1.6% in clot recall and 2.8% in negative screening).

CONCLUSIONS

The AI system enhances diagnostic performance and efficiency of CTPA interpretation by first-on-call residents, substantially reducing missed PE and narrowing experience-based performance gaps.