Diagnostic and interventional imaging · 2024
Detection and characterization of pancreatic lesion with artificial intelligence: The SFR 2023 artificial intelligence data challenge.
СтатьяМногоцентровое исследованиеНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Aouad Theodore, Laurent Valerie, Levant Paul, Rode Agnes, Brillat-Savarin Nina, Gaillot Pénélope, Hoeffel Christine, Frampas Eric, Barat Maxime, Russo Roberta, Wagner Mathilde, Zappa Magaly, Ernst Olivier, Delagnes Anais, Fillias Quentin, Dawi Lama, Savoye-Collet Céline, Copin Pauline, Calame Paul, Reizine Edouard, Luciani Alain, Bellin Marie-France, Talbot Hugues, Lassau Nathalie
Аннотация
PURPOSE: The purpose of the 2023 SFR data challenge was to invite researchers to develop artificial intelligence (AI) models to identify the presence of a pancreatic mass and distinguish between benign and malignant pancreatic masses on abdominal computed tomography (CT) examinations.
MATERIALS AND METHODS: Anonymized abdominal CT examinations acquired during the portal venous phase were collected from 18 French centers. Abdominal CT examinations were divided into three groups including CT examinations with no lesion, CT examinations with benign pancreatic mass, or CT examinations with malignant pancreatic mass. Each team included at least one radiologist, one data scientist, and one engineer. Pancreatic lesions were annotated by expert radiologists. CT examinations were distributed in balanced batches via a Health Data Hosting certified platform. Data were distributed into four batches, two for training, one for internal evaluation, and one for the external evaluation. Training used 83 % of the data from 14 centers and external evaluation used data from the other four centers. The metric (i.e., final score) used to rank the participants was a weighted average of mean sensitivity, mean precision and mean area under the curve.
RESULTS: A total of 1037 abdominal CT examinations were divided into two training sets (including 500 and 232 CT examinations), an internal evaluation set (including 139 CT examinations), and an external evaluation set (including 166 CT examinations). The training sets were distributed on September 7 and October 13, 2023, and evaluation sets on October 15, 2023. Ten teams with a total of 93 members participated to the data challenge, with the best final score being 0.72.
CONCLUSION: This SFR 2023 data challenge based on multicenter CT data suggests that the use of AI for pancreatic lesions detection is possible on real data, but the distinction between benign and malignant pancreatic lesions remains challenging.
MATERIALS AND METHODS: Anonymized abdominal CT examinations acquired during the portal venous phase were collected from 18 French centers. Abdominal CT examinations were divided into three groups including CT examinations with no lesion, CT examinations with benign pancreatic mass, or CT examinations with malignant pancreatic mass. Each team included at least one radiologist, one data scientist, and one engineer. Pancreatic lesions were annotated by expert radiologists. CT examinations were distributed in balanced batches via a Health Data Hosting certified platform. Data were distributed into four batches, two for training, one for internal evaluation, and one for the external evaluation. Training used 83 % of the data from 14 centers and external evaluation used data from the other four centers. The metric (i.e., final score) used to rank the participants was a weighted average of mean sensitivity, mean precision and mean area under the curve.
RESULTS: A total of 1037 abdominal CT examinations were divided into two training sets (including 500 and 232 CT examinations), an internal evaluation set (including 139 CT examinations), and an external evaluation set (including 166 CT examinations). The training sets were distributed on September 7 and October 13, 2023, and evaluation sets on October 15, 2023. Ten teams with a total of 93 members participated to the data challenge, with the best final score being 0.72.
CONCLUSION: This SFR 2023 data challenge based on multicenter CT data suggests that the use of AI for pancreatic lesions detection is possible on real data, but the distinction between benign and malignant pancreatic lesions remains challenging.