Grad-CAM based deep learning analytics for image-level colon disease classification based on graph neural networks and vision transformers.

INTRODUCTION

Accurate classification of colonoscopic images is essential for early detection and characterization of colorectal diseases. Recent advances in deep learning, particularly transformer-based architectures and graph neural networks (GNNs), provide alternative strategies for modeling global contextual information and relational structures in image representations. This study evaluates transformer-based and graph-based frameworks under a unified experimental protocol for endoscopic colon disease classification.

METHODS

Experiments were conducted on the Kvasir V2 dataset using two primary paradigms: (i) a Vision Transformer (ViT) with selective fine-tuning and learning-rate scheduling, and (ii) a CNN-GNN pipeline integrating image embeddings with graph construction strategies (cosine similarity, k-nearest neighbors, and epsilon-radius graphs) and multiple GNN architectures. Performance was evaluated using accuracy, precision, recall, and macro-F1 score, with Grad-CAM used for qualitative interpretability analysis.

RESULTS

The selectively fine-tuned Vision Transformer achieved 94.6% accuracy with a macro-F1 score of 0.94. The best graph-based configuration (ViT embeddings with epsilon graph and GIN aggregation) achieved 92% accuracy and 0.92 macro-F1 score.

DISCUSSION: Transformer-based contextual modeling provides strong discriminative capability for image-level colon disease classification, while graph-based relational modeling offers competitive performance when paired with high-quality embeddings.