A Lightweight Network With Uncertainty-Guided Latent Space Refinement for Multi-Modal Brain Tissue and Tumor Extraction.

Brain tissue and tumor extraction plays a pivotal role in medical care and clinical research. Leveraging the diverse and complementary information provided by different imaging modalities is crucial to the success of these applications. However, existing deep learning-based methods exhibit limitations in two major aspects. First, they ignore the incorporation of task-oriented regularization when fusing multi-modal features, leading to suboptimal latent space learning. Second, these methods tend to overlook the explicit modeling and exploitation of prediction uncertainty, despite the strong correlation between prediction uncertainty and errors. To address these issues, we propose a novel lightweight network with uncertainty-guided latent space refinement for multi-modal brain tissue and tumor extraction, called UMNet. Particularly, UMNet features a modality-specific uncertainty-regularized feature fusion module (M-SUM), which facilitates latent space refinement to enable a more informed and effective aggregation of complementary multi-modal information. Additionally, we design an uncertainty-enhanced loss function (U-Loss) to explicitly harness the connection between prediction uncertainty and errors. Experimental results demonstrate that our proposed UMNet achieves promising performance, outperforming state-of-the-art methods for both brain tissue and tumor extraction tasks.