Concurrent AI-human interaction in prostate cancer MRI interpretation: More hype than help?

OBJECTIVE

We evaluated a commercial artificial intelligence (AI) system as a concurrent decision-support tool for clinically significant prostate cancer (csPCa) detection.

MATERIALS AND

METHODS

In our retrospective study, consecutive patients underwent multiparametric MRI for clinical suspicion of PCa. All scans were reviewed by six readers with varying expertise (two expert radiologists, > 1,000 cases; two basic radiologists, 400‒1,000 cases; and two residents), with and without AI assistance. Intra-/inter-reader agreements and the impact of AI-assistance on patient-level csPCa scores and diagnostic performance, as well as benefit-to-harm ratios, were assessed.

RESULTS

The population consisted of 100 patients with a 26% prevalence of csPCa. There was no improvement in inter-reader agreement with AI-assistance versus without (Fleiss κ 0.573 and 0.584, respectively). Residents were most likely to change PI-RADS scores on AI-assisted readings compared to basic and expert radiologists (19, 9, and 7 changes, respectively). Overall, there was no significant difference in area under the receiving operating characteristic curve between AI-assisted and AI-unassisted readings (0.87 versus 0.86; p = 0.734). At a PI-RADS ≥ 3 threshold, sensitivity was slightly lower with AI (0.87 versus 0.89), while specificity (0.73), positive predictive value (0.53-0.54), and negative predictive value (0.94-0.95) remained similar. Subgroup analyses showed no significant differences in diagnostic performance. A slight increase in grade selectivity and selective biopsy avoidance rate was observed among experts and residents, respectively, with AI-assisted readings when applying a PI-RADS cutoff of 3 or PSA density ≥ 0.15 ng/mL/mL.

CONCLUSIONS

AI did not significantly improve diagnostic accuracy across readers of varying expertise, with minor impacts on benefit-to-harm ratios.

RELEVANCE STATEMENT: We found that AI support in prostate MRI did not significantly improve diagnostic accuracy across readers of varying experience, highlighting the need for further research to optimize AI integration and define its most clinically meaningful roles in prostate cancer detection.

KEY POINTS: Residents were most prone to PI-RADS score modifications after AI-assisted readings compared to AI-unassisted and expert readers. There was no significant difference in diagnostic performance metrics between AI-assisted and unassisted readings. A slight improvement in grade selectivity among experts and in selective biopsy avoidance among residents was observed during AI-assisted readings for biopsy recommendations.