Computer methods and programs in biomedicine · 2026
MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Shang Lin Wei, Guo Ying Xi, Li Yun Hong, Zhu Jian Bo, Ji Xin Yi, Xiong Shuang, Wu Jin Jin, Huang Xing, Wang Hui Jie, Yin Jian Hua
Аннотация
BACKGROUND AND OBJECTIVE: Virtual staining is based on label-free imaging technology and deep learning, which enables the efficient and non-destructive generation of virtual stained images with effects similar to those of traditional pathological staining, thus holding great application potential in future clinical medical practice. However, the information provided by a single imaging technology is limited, which is insufficient to support high-accuracy virtual staining.
METHODS: This study took breast cancer tissue as the research object. Bright-field, auto-fluorescence and phase contrast imaging technology were introduced to simultaneously extract the multimodal complementary information. A multi-branch input network was established to fuse the three types of optical images and generate high-accuracy virtual stained images. To enhance the model's performance, three strategies were employed, including designing a Triple-symmetric cross-attention module to handle the interactive information of multimodal image features, introducing multi-scale discriminator to ensure the model's ability to capture both global features and local details, and applying transfer-learning to address the gradient dilution issue of multi-branches.
RESULTS: Through the evaluation of various image similarity metrics and nuclear statistical metrics, the virtual stained images based on multimodal fusion achieved high accuracy and were significantly superior to any single-modal virtual staining approach. Further ablation experiments explained the reason of the aforementioned results and quantitatively demonstrated the improvements brought by the three optimization strategies.
CONCLUSIONS: This work advances the application of multimodal fusion in virtual staining, address the key challenges of cross-modal interactive information mining and multi-branch model training, and further enhance the accuracy and reliability of virtual staining.
METHODS: This study took breast cancer tissue as the research object. Bright-field, auto-fluorescence and phase contrast imaging technology were introduced to simultaneously extract the multimodal complementary information. A multi-branch input network was established to fuse the three types of optical images and generate high-accuracy virtual stained images. To enhance the model's performance, three strategies were employed, including designing a Triple-symmetric cross-attention module to handle the interactive information of multimodal image features, introducing multi-scale discriminator to ensure the model's ability to capture both global features and local details, and applying transfer-learning to address the gradient dilution issue of multi-branches.
RESULTS: Through the evaluation of various image similarity metrics and nuclear statistical metrics, the virtual stained images based on multimodal fusion achieved high accuracy and were significantly superior to any single-modal virtual staining approach. Further ablation experiments explained the reason of the aforementioned results and quantitatively demonstrated the improvements brought by the three optimization strategies.
CONCLUSIONS: This work advances the application of multimodal fusion in virtual staining, address the key challenges of cross-modal interactive information mining and multi-branch model training, and further enhance the accuracy and reliability of virtual staining.