The British journal of radiology · 2025
Real world clinical experience of using Brainomix e-CTA software in a medium size acute National Health Service Trust.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Merchant Fraser, Choulerton James, James Richard, Pang Chun Lap
Аннотация
OBJECTIVES: Artificial intelligence (AI) software including Brainomix "e-CTA" which detect large vessel occlusions (LVO) have clinical potential. We hypothesized that in real world use where prevalence is low, its clinical utility may be overstated.
METHODS: In this single centre retrospective service evaluation project, data sent to Brainomix from a medium size acute National Health Service (NHS) Trust hospital between January 3, 2022 and January 3, 2023 was reviewed. 584 intracranial computed tomography angiogram (CTA) datasets were analysed for LVO by e-CTA. The e-CTA output and radiology report were compared to ground truth, defined by a consultant radiologist with fellowship neuroradiology training, with access to subsequent imaging and clinical notes. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated.
RESULTS: Of 584 cases (45% female, mean age 70 ± 16 years), 9% (n = 50) had LVO. e-CTA had a sensitivity of 0.78 (95% CI 0.64-0.88), specificity of 0.93 (0.9-0.95), PPV of 0.5 (0.42-0.58), and NPV of 0.98 (0.96-0.99). e-CTA had an error rate of 9% (52/584). Erroneous cases were categorized into causes for error. Common causes for false positives included incorrect anatomy (21%, 8/39) and other pathology (13%, 5/39), with several uncategorizable cases (39%, 15/39). Common causes for false negatives included LVO within the terminal internal carotid artery (ICA) (55%, 6/11) and uncategorizable (18%, 2/11).
CONCLUSIONS: We demonstrated that PPV of e-CTA is poor in consecutive cases in a real-world NHS setting. We advocate for local validation of AI software prior to clinical use.
ADVANCES IN KNOWLEDGE: Common AI errors were due to anatomical misidentification, presence of other pathology, and misidentifying LVO in the terminal ICA.
METHODS: In this single centre retrospective service evaluation project, data sent to Brainomix from a medium size acute National Health Service (NHS) Trust hospital between January 3, 2022 and January 3, 2023 was reviewed. 584 intracranial computed tomography angiogram (CTA) datasets were analysed for LVO by e-CTA. The e-CTA output and radiology report were compared to ground truth, defined by a consultant radiologist with fellowship neuroradiology training, with access to subsequent imaging and clinical notes. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated.
RESULTS: Of 584 cases (45% female, mean age 70 ± 16 years), 9% (n = 50) had LVO. e-CTA had a sensitivity of 0.78 (95% CI 0.64-0.88), specificity of 0.93 (0.9-0.95), PPV of 0.5 (0.42-0.58), and NPV of 0.98 (0.96-0.99). e-CTA had an error rate of 9% (52/584). Erroneous cases were categorized into causes for error. Common causes for false positives included incorrect anatomy (21%, 8/39) and other pathology (13%, 5/39), with several uncategorizable cases (39%, 15/39). Common causes for false negatives included LVO within the terminal internal carotid artery (ICA) (55%, 6/11) and uncategorizable (18%, 2/11).
CONCLUSIONS: We demonstrated that PPV of e-CTA is poor in consecutive cases in a real-world NHS setting. We advocate for local validation of AI software prior to clinical use.
ADVANCES IN KNOWLEDGE: Common AI errors were due to anatomical misidentification, presence of other pathology, and misidentifying LVO in the terminal ICA.