Development of Artificial Intelligence-based Real-time Automatic Fusion of Multiparametric Magnetic Resonance Imaging and Transrectal Ultrasonography of the Prostate.

OBJECTIVE

To report the development of artificial intelligence (AI)-based software to allow for the autonomous fusion of transrectal ultrasound and multiparametric magnetic resonance images of the prostate to be used during transperineal prostate biopsies.

MATERIALS AND

METHODS

This study was approved by the Institutional Review Board (protocol ID 3167CESC). The automatic software development for fusion biopsy involved 3 steps: (1) developing an AI component to segment the prostate during ultrasound; (2) developing the component to segment anatomical structures in magnetic resonance images using labeled datasets from the Cancer Imaging Archive and in-house scans; (3) developing the fusion component to register segmented ultrasound and magnetic resonance images using a 3-step process: pre-alignment, rigid alignment, and elastic fusion, validated by measuring the lesion distance between modalities. Statistical analysis included descriptive statistics and the Mann-Whitney U test, evaluating outcomes with Dice scores and average precision metrics.

RESULTS

The ultrasound component showed a Dice score of 0.87 with a test set of 75,357 images and 28,946 annotations. The magnetic resonance component achieved a Dice score of 0.85 on a test set of 2494 images and annotations. It also demonstrated a mean average precision of 0.80 for bounding boxes and 0.88 for segmented objects, both measured at a 50% intersection over union threshold. The fusion AI component reduced the median magnetic resonance-ultrasound lesion distance from 8 mm (interquartile ranges 6-9) after rigid fusion to 4 mm (interquartile ranges 3-5) after elastic fusion (P <.001).

CONCLUSION

A data processing pipeline and AI were created to allow for the autonomous fusion of ultrasound and magnetic resonance images to be ideally used during transperineal prostate biopsies.