Cascaded neural network segmentation pipeline for automated delineation of prostate and organs at risk in male pelvic CT.

Delineation of the prostate and nearby organs at risk (OARs) is a fundamental step in prostate cancer radiation therapy planning. Such contouring is often done manually, which can be a time-consuming and highly variable process. To alleviate these issues, we propose a fully automated two-step deep learning approach to segment the prostate, bladder, rectum, seminal vesicles, and femoral heads from CT images. The first step localizes the organs of interest using a modified 3D UNet architecture that contains an axial cross-attention module. Final segmentations are then computed for each organ individually using organ-specifically optimized UNet-based models. A total of 275 CT images were used for model training and validation. When evaluated on a hold-out set of 15 image sets, the full pipeline achieved mean dice similarity coefficients (DSC) and 95% Hausdorff distances (95HD, in mm) of 0.866±0.034 and 4.46±1.02 (prostate), 0.957±0.014 and 2.91±0.289 (bladder), 0.853±0.044 and 5.10±1.87 (rectum), 0.740±0.117 and 6.72±9.46 (seminal vesicles), 0.942±0.016 and 2.85±1.04 (left femoral head), 0.942±0.018 and 3.04±1.37 (right femoral head).