Reliability and validity of an artificial intelligence-assisted system for the detection of abnormalities in chest and bone radiographs in an emergency department.

OBJECTIVE

To evaluate the diagnostic performance of two commercial artificial intelligence (AI) systems-ChestView for chest radiographs (CXR) and BoneView for bone radiographs (BXR)-in an emergency department (ED), and compare their validity with that of observers with different professional profiles and levels of experience: emergency physicians, radiology trainees, and expert radiologists.

METHODS

We conducted a diagnostic test evaluation study on a random selection of 346 CXRs and 261 BXRs requested in the ED. Examinations were independently analysed by the AI systems and the various observers. The reference diagnosis (gold standard) was established by consensus among 3 radiologists, resorting to additional imaging tests or clinical information when necessary. Sensitivity, specificity, and positive and negative (NPV) predictive values were then calculated and compared.

RESULTS

For CXRs, AI (ChestView) showed overall sensitivity (64.4%) significantly higher than that of emergency physicians (49.2%; P = .018), although lower than that of the expert radiologist (83.9%; P .001). Performance was notable for the detection of nodules/masses (sensitivity 80.0%) and pneumothorax (NPV, 99.7%), but lower for consolidations (sensitivity, 40.4%). For BXRs, AI (BoneView) achieved sensitivity for fracture detection (87.5%) higher than that of the expert radiologist (77.1%), with an NPV of 96.9%. However, its performance was lower for detecting dislocations (sensitivity 60.0%) and joint effusions (25.0%).

CONCLUSIONS

The evaluated AI systems demonstrate clinically relevant performance in the emergency setting, significantly enhancing the diagnostic capacity of emergency physicians. Their high sensitivity for fracture detection and high NPV for pulmonary nodules, pneumothorax, and fractures establish them as a high-impact safety tool.