Artificial intelligence in emergency skeletal X-ray: post-deployment monitoring and clinical impact of incorrect AI results.

OBJECTIVES

This study evaluates the impact of the artificial intelligence (AI) application BoneViewTM within a radiographer-supervised clinical workflow and subsequent effects on patient management. We examine false AI findings to understand the clinical consequences in skeletal X-ray examinations of suspected fracture injuries.

MATERIALS &

METHODS

This retrospective study was conducted at Bærum Hospital, Norway, where BoneViewTM was introduced within an AI-assisted radiographer-supervised emergency skeletal X-ray workflow in September 2023. The study included patients who were referred to emergency skeletal X-ray in January 2024. We calculated diagnostic accuracy of AI compared with AI-assisted radiologists, and we examined patient management following the introduction of the new AI-assisted workflow.

RESULTS

The study included 1248 patients. AI alone demonstrated an overall sensitivity of 0.95 (95% CI 0.92 to 0.97) and specificity of 0.90 (95% CI 0.88 to 0.92) when compared to AI-assisted radiologists. Compared to AI's initial suggestion, radiographer supervision altered the subsequent patient pathway in 20% of the cases, with the likelihood overriding AI being higher when AI results were later found to be incorrect (i.e. false positives or false negatives). Notably, the AI-assisted workflow only led to premature discharge of one patient who required treatment, and this patient was recalled to hospital for treatment following the radiologist's report.

CONCLUSION

AI demonstrates strong diagnostic accuracy with high sensitivity and specificity in emergency skeletal X-ray examinations. Radiographers may play a crucial role in mitigating erroneous clinical decisions due to false AI-results and ensure appropriate patient management.