MedElevator Войти в исследования
← Поиск публикаций
Медицинская библиотека · ArXiv · 2026

Current validation practice undermines surgical AI development.

Reinke Annika, Li Ziying O, Tizabi Minu D, André Pascaline, Knopp Marcel, Rother Mika M, Machado Ines P, Altieri Maria S, Alapatt Deepak, Bano Sophia, Bodenstedt Sebastian, Burgert Oliver, Chen Elvis C S, Collins Justin W, Colliot Olivier, Christodoulou Evangelia, Czempiel Tobias, Das Adrito, Docea Reuben, Donoho Daniel, Dou Qi, Eckhoff Jennifer, Engelhardt Sandy, Fichtinger Gabor, Fuernstahl Philipp, Kilroy Pablo García, Giannarou Stamatia, Gilbert Stephen, Gockel Ines, Godau Patrick, Gödeke Jan, Grantcharov Teodor P, Haidegger Tamas, Hann Alexander, Hashizume Makoto, Heitz Charles, Hisey Rebecca, Hoffmann Hanna, Huaulmé Arnaud, Jäger Paul F, Jannin Pierre, Jarc Anthony, Jena Rohit, Jin Yueming, Joskowicz Leo, Joyeux Luc, Kirchner Max, Krieger Axel, Kronreif Gernot, Lam Kyle, Laufer Shlomi, Lavanchy Joël L, Lee Gyusung I, Lim Robert, Liu Peng, Marcus Hani J, Mascagni Pietro, Meireles Ozanan R, Mueller Beat P, Mündermann Lars, Nakawala Hirenkumar, Navab Nassir, Ndong Abdourahmane, Neumann Juliane, Nickel Felix, Nolden Marco, Nwoye Chinedu, Oh Namkee, Padoy Nicolas, Pausch Thomas, Pfeiffer Micha, Rädsch Tim, Ren Hongliang, Rieke Nicola, Rivoir Dominik, Sarikaya Duygu, Schmidgall Samuel, Seibold Matthias, Seidlitz Silvia, Seitel Alexander, Sharan Lalith, Siewerdsen Jeffrey H, Srivastav Vinkle, Sznitman Raphael, Taylor Russell, Tran Thuy N, Unberath Matthias, van der Sommen Fons, Wagner Martin, Yamlahi Amine, Zhou Shaohua K, Zia Aneeq, Madani Amin, Stoyanov Danail, Speidel Stefanie, Hashimoto Daniel A, Kolbinger Fiona R, Maier-Hein Lena

СтатьяПрепринтПеревод в очереди

Аннотация

Surgical data science (SDS) is rapidly advancing, yet clinical adoption of artificial intelligence (AI) in surgery remains limited, with inadequate validation emerging as an important contributing factor. In fact, existing validation practices often neglect the temporal and hierarchical structure of intraoperative videos, producing misleading, unstable, or clinically irrelevant results. In a pioneering, consensus-driven effort, we introduce a comprehensive catalog of validation pitfalls in AI-based surgical video analysis that was derived from a multi-stage Delphi process with 92 international experts. The collected pitfalls span three categories: (1) data (e.g., incomplete annotation, spurious correlations), (2) metric selection and configuration (e.g., neglect of temporal stability, mismatch with clinical needs), and (3) aggregation and reporting (e.g., clinically uninformative aggregation, failure to account for frame dependencies in hierarchical data structures). A systematic review of surgical AI papers reveals that these pitfalls are widespread in current practice, with the majority of studies failing to account for temporal dynamics or hierarchical data structure, or relying on clinically uninformative metrics. Experiments on real surgical video datasets provide empirical evidence that ignoring temporal and hierarchical data structures can substantially understate uncertainty, obscure critical failure modes, and even alter algorithm rankings. To address these shortcomings, we provide a catalogue of best practices compiled in a multi-stage Delphi process. Together, this work provides an evidence-based framework to inform more rigorous validation of surgical video analysis algorithms and to guide future efforts in benchmarking, reporting, regulatory review, and clinical translation.

Для цитирования: Reinke Annika, Li Ziying O, Tizabi Minu D, André Pascaline, Knopp Marcel, Rother Mika M, Machado Ines P, Altieri Maria S, Alapatt Deepak, Bano Sophia, Bodenstedt Sebastian, Burgert Oliver, Chen Elvis C S, Collins Justin W, Colliot Olivier, Christodoulou Evangelia, Czempiel Tobias, Das Adrito, Docea Reuben, Donoho Daniel, Dou Qi, Eckhoff Jennifer, Engelhardt Sandy, Fichtinger Gabor, Fuernstahl Philipp, Kilroy Pablo García, Giannarou Stamatia, Gilbert Stephen, Gockel Ines, Godau Patrick, Gödeke Jan, Grantcharov Teodor P, Haidegger Tamas, Hann Alexander, Hashizume Makoto, Heitz Charles, Hisey Rebecca, Hoffmann Hanna, Huaulmé Arnaud, Jäger Paul F, Jannin Pierre, Jarc Anthony, Jena Rohit, Jin Yueming, Joskowicz Leo, Joyeux Luc, Kirchner Max, Krieger Axel, Kronreif Gernot, Lam Kyle, Laufer Shlomi, Lavanchy Joël L, Lee Gyusung I, Lim Robert, Liu Peng, Marcus Hani J, Mascagni Pietro, Meireles Ozanan R, Mueller Beat P, Mündermann Lars, Nakawala Hirenkumar, Navab Nassir, Ndong Abdourahmane, Neumann Juliane, Nickel Felix, Nolden Marco, Nwoye Chinedu, Oh Namkee, Padoy Nicolas, Pausch Thomas, Pfeiffer Micha, Rädsch Tim, Ren Hongliang, Rieke Nicola, Rivoir Dominik, Sarikaya Duygu, Schmidgall Samuel, Seibold Matthias, Seidlitz Silvia, Seitel Alexander, Sharan Lalith, Siewerdsen Jeffrey H, Srivastav Vinkle, Sznitman Raphael, Taylor Russell, Tran Thuy N, Unberath Matthias, van der Sommen Fons, Wagner Martin, Yamlahi Amine, Zhou Shaohua K, Zia Aneeq, Madani Amin, Stoyanov Danail, Speidel Stefanie, Hashimoto Daniel A, Kolbinger Fiona R, Maier-Hein Lena. Current validation practice undermines surgical AI development.. ArXiv. 2026.
Подготовка и перевод публикации
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…

История · было → стало

Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.

Тип публикации указан по данным источника. Уровень доказательности автоматически не присваивается.