Impact of AI Assistance in Pneumothorax Detection on Chest Radiographs Among Readers of Varying Experience.

Objectives

We aimed to investigate whether AI assistance could improve the performance of pneumothorax detection on chest radiographs (CXR) by readers with varying experience from radiologists to the frontline healthcare providers, and whether AI assistance could diminish the potential confounders for readers' detecting pneumothorax.

Methods

In this retrospective, single-center, blinded, multi-reader diagnostic accuracy study, 125 CXRs were prepared from radiological information system (March 2024 to August 2024) for test. The 18 readers were composed of six groups, each had 3 persons: board-certified radiologists (Group-1), senior radiology residents (Group-2), junior radiology residents (Group-3), postgraduate year residents (Group-4), senior radiographers (Group-5), and junior radiographers (Group-6). They read the CXR independently twice, without and with AI assistance, at an interval of one month. We used receiver operating characteristic curve for performance analysis and generalized estimating equation (GEE) model for confounding factor analysis.

Results

AI software alone achieved a high area under curve of 0.965 (95% CI: 0.926, 0.995). With AI assistance, the performance in all groups significantly improved (p < 0.01) especially the junior readers (the frontline healthcare providers, Group-3, 4, 6) and diminished the difference among all groups except some related to Group-1. GEE model showed that AI assistance, reader's experience, and projection type interfere with the readers' performance (all p < 0.05).

Conclusions

AI assistance could improve the performance of pneumothorax detection by varying experience of readers, especially the frontline healthcare providers. The influence of confounders, such as reader's experience, also be diminished by AI assistance.