Physics and imaging in radiation oncology · 2025
Establishing prospective performance monitoring for real-world implementation of deep learning-based auto-segmentation in prostate cancer radiotherapy.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Zhu Libing, Rong Yi, Yu Nathan Y, Holmes Jason M, Vargas Carlos E, James Sarah E, Shang Lu, Rwigema Jean-Claude M, Chen Quan
Аннотация
BACKGROUND AND PURPOSE: Deep-learning auto-segmentation (DLAS) performance in radiotherapy may change over time due to data shift/drift or practice changes, yet guidance for quality assurance is lacking. This study developed a practical framework for prospective performance monitoring using retrospective data.
METHODS: A total of 464 prostate cases over 20 months were retrospectively collected. Two commercial DLAS models were clinically used: model A (2D U-Net, January 2022-January 2023) and model B (3D U-Net, February-August 2023). The agreement between DLAS and clinical contours was assessed using Dice Similarity Coefficient (DSC), 95th percentile Hausdorff Distance (HD95), and Surface DSC with a 2 mm tolerance (SDSC). Statistical process control charts were created to monitor performance drift and model switching. The first 150 cases were used to define organ-specific control limits with two and three standard deviations of monthly mean values, σ x ¯ .
RESULTS: 2 σ x ¯ and 3 σ x ¯ -based control limits were established for the monthly average charts, ranging from DSC 0.82-0.97, HD95 1.4-10.5 mm, and SDSC 0.45-0.91 across organs. Model A showed stable performance, with 9-13 months per organ remaining within the 3 σ x ¯ thresholds. In contrast, model B demonstrated a marked performance shift (p < 0.001), with all five organs exceeding both thresholds across all 7 months. The 2 σ x ¯ thresholds were more sensitive in detecting mild deviations for model A, while both limits effectively identified the substantial drift of model B.
CONCLUSION: The monitoring system effectively detected out-of-distribution outliers and clinical practice changes, providing a reliable framework for early detection of monthly performance degradation.
METHODS: A total of 464 prostate cases over 20 months were retrospectively collected. Two commercial DLAS models were clinically used: model A (2D U-Net, January 2022-January 2023) and model B (3D U-Net, February-August 2023). The agreement between DLAS and clinical contours was assessed using Dice Similarity Coefficient (DSC), 95th percentile Hausdorff Distance (HD95), and Surface DSC with a 2 mm tolerance (SDSC). Statistical process control charts were created to monitor performance drift and model switching. The first 150 cases were used to define organ-specific control limits with two and three standard deviations of monthly mean values, σ x ¯ .
RESULTS: 2 σ x ¯ and 3 σ x ¯ -based control limits were established for the monthly average charts, ranging from DSC 0.82-0.97, HD95 1.4-10.5 mm, and SDSC 0.45-0.91 across organs. Model A showed stable performance, with 9-13 months per organ remaining within the 3 σ x ¯ thresholds. In contrast, model B demonstrated a marked performance shift (p < 0.001), with all five organs exceeding both thresholds across all 7 months. The 2 σ x ¯ thresholds were more sensitive in detecting mild deviations for model A, while both limits effectively identified the substantial drift of model B.
CONCLUSION: The monitoring system effectively detected out-of-distribution outliers and clinical practice changes, providing a reliable framework for early detection of monthly performance degradation.