Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics (AIFB) · 2025
Performance of multi-vendor auto-segmentation models for thoracic organs at risk trained on a single dataset.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Emin Sevgi, Rossi Elia, Hedman Mattias, Giovenco Marcela, Villegas Fernanda, Onjukka Eva
Аннотация
INTRODUCTION: This study evaluates the delineation quality of artificial intelligence (AI)-based models for auto-segmentation trained on the same dataset, as the intrinsic performance cannot be evaluated for commercial solutions due to differences in training datasets. A diverse set of challenging thoracic organs-at-risk (OAR) were chosen, to reveal potential limitations of AI-based tools which are relevant for their clinical adoption.
MATERIALS & METHODS: A structure set with 16 OAR was delineated and reviewed by radiation oncology experts for 250 patients with lung tumours (200/50 for training/testing). Three participating vendors had access to the training dataset for a limited time to develop a model mimicking their commercial model development strategies. The models were tested on the blind test dataset by the authors. A quantitative analysis was performed employing Dice Similarity Coefficient (DSC), surface DSC (sDSC), the 95-th percentile of the Hausdorff Distance (HD95) and average symmetric surface distance (ASSD). Inter-observer variability in manual segmentation was estimated by three independent expert delineations for a subset of five test patients.
RESULTS: 13 OAR had DSC > 0.8, 9 had sDSC > 0.8, 10 had ASSD < 0.5 mm and 5 had HD95 < 1 mm. The most challenging structures to auto-segment were the brachial plexus, pulmonary vein, and vena cava inferior. The overall results for all models were exceeding the inter-observer variability for all metrics.
CONCLUSION: While the evaluated AI-models perform very well for some OAR, they appear less successful at modelling organs with branching structures and poor image contrast, even when trained on a large homogeneous dataset.
MATERIALS & METHODS: A structure set with 16 OAR was delineated and reviewed by radiation oncology experts for 250 patients with lung tumours (200/50 for training/testing). Three participating vendors had access to the training dataset for a limited time to develop a model mimicking their commercial model development strategies. The models were tested on the blind test dataset by the authors. A quantitative analysis was performed employing Dice Similarity Coefficient (DSC), surface DSC (sDSC), the 95-th percentile of the Hausdorff Distance (HD95) and average symmetric surface distance (ASSD). Inter-observer variability in manual segmentation was estimated by three independent expert delineations for a subset of five test patients.
RESULTS: 13 OAR had DSC > 0.8, 9 had sDSC > 0.8, 10 had ASSD < 0.5 mm and 5 had HD95 < 1 mm. The most challenging structures to auto-segment were the brachial plexus, pulmonary vein, and vena cava inferior. The overall results for all models were exceeding the inter-observer variability for all metrics.
CONCLUSION: While the evaluated AI-models perform very well for some OAR, they appear less successful at modelling organs with branching structures and poor image contrast, even when trained on a large homogeneous dataset.