Multi-resolution and Multi-modal Feature Integration using Graph Neural Networks for Optical Coherence Tomography Image Classification.

Early and accurate diagnosis of retinal diseases is crucial, as many of these conditions can be prevented or managed effectively with timely treatment. Optical Coherence Tomography (OCT) is a widely used medical imaging technique that provides high-resolution visualization of the retina, allowing the detection of abnormalities. Current state-of-the-art methods predominantly rely on extracting either local or global features using Convolutional Neural Networks or Vision Transformers from OCT images. Moreover, few existing ensemble methods primarily focus on concatenation and decision-level feature fusion, limiting their ability to fully capture the relationships between distinct feature types. In this study, we propose to incorporate multi-modal semantic features extracted at multiple feature resolutions to achieve better classification of OCT images. In addition, to effectively fuse the multitude of extracted information, a learnable graph-based effective feature fusion method is proposed. This allows us to effectively capture complex lesion patterns and subtle abnormalities in the retina. We evaluate our method using three publicly available datasets: OCTDL, OCTID, and OCT2014 and surpass the performance of current state-of-the-art models with significant margins, demonstrating the robustness and reliability of our approach for retinal disease classification.