BJR artificial intelligence · 2026
Federated artificial intelligence monitoring service (FAMOS): an in silico feasibility study.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Luis Aysha, Scarsbrook Andrew, Grzeda Mariusz, Lampignano Jesus Perdomo, Clark Matt, Wheller Bob, Baldwin Jack, Cairns James, Dhesi Simran, Hall Mark, Lowe David J, Shuaib Haris
Аннотация
OBJECTIVES: To evaluate feasibility of a federated AI monitoring service (FAMOS) for post-deployment surveillance of third-party AI applications used in chest X-ray (CXR) interpretation.
METHODS: FAMOS was deployed at 2 NHS Trusts using a federated architecture enabling local data processing while maintaining data governance compliance. De-identified CXRs from patients aged >18 years were retrospectively identified, along with relevant patient attributes (age, sex, inpatient status, image orientation, season, artefact). Chest X-rays were processed by 2 AI applications to simulate real-world deployment. FAMOS analyzed input data, AI inference values, and longitudinal human-AI agreement as proxy indicators of data, prediction, and behavioral drift. Input monitoring used image feature embeddings analyzed with principal component analysis and Hotelling's T² statistics, while risk-adjusted cumulative sum charts were applied to sequentially monitor AI inference values. human-AI agreement was evaluated at 10 time points over 3 months.
RESULTS: Input monitoring identified a small proportion of outlier examinations, predominantly associated with modifiable image quality issues. Artificial intelligence inference values remained stable across most findings for both vendors, with limited drift events detected. Human-AI agreement patterns differed between sites, remaining stable at 1 site while increasing over time at another, suggesting evolving automation bias.
CONCLUSIONS: Real-time, federated monitoring of deployed radiology AI systems is technically and operationally feasible within clinical environments. Multi-domain platform-based monitoring provides scalable, independent oversight and may function as an early-warning system supporting identification of emerging risks following deployment.
ADVANCES IN KNOWLEDGE: This study introduces a federated, multi-domain monitoring framework integrating sociotechnical indicators for continuous post-deployment surveillance of clinical radiology AI tools.
METHODS: FAMOS was deployed at 2 NHS Trusts using a federated architecture enabling local data processing while maintaining data governance compliance. De-identified CXRs from patients aged >18 years were retrospectively identified, along with relevant patient attributes (age, sex, inpatient status, image orientation, season, artefact). Chest X-rays were processed by 2 AI applications to simulate real-world deployment. FAMOS analyzed input data, AI inference values, and longitudinal human-AI agreement as proxy indicators of data, prediction, and behavioral drift. Input monitoring used image feature embeddings analyzed with principal component analysis and Hotelling's T² statistics, while risk-adjusted cumulative sum charts were applied to sequentially monitor AI inference values. human-AI agreement was evaluated at 10 time points over 3 months.
RESULTS: Input monitoring identified a small proportion of outlier examinations, predominantly associated with modifiable image quality issues. Artificial intelligence inference values remained stable across most findings for both vendors, with limited drift events detected. Human-AI agreement patterns differed between sites, remaining stable at 1 site while increasing over time at another, suggesting evolving automation bias.
CONCLUSIONS: Real-time, federated monitoring of deployed radiology AI systems is technically and operationally feasible within clinical environments. Multi-domain platform-based monitoring provides scalable, independent oversight and may function as an early-warning system supporting identification of emerging risks following deployment.
ADVANCES IN KNOWLEDGE: This study introduces a federated, multi-domain monitoring framework integrating sociotechnical indicators for continuous post-deployment surveillance of clinical radiology AI tools.