Interpreting Deep Neural Networks in Diabetic Retinopathy Grading: A Comparison with Human Decision Criteria.

Diabetic retinopathy (DR) causes visual impairment and blindness in millions of diabetic patients globally. Fundus image-based Automatic Diabetic Retinopathy Classifiers (ADRCs) can ensure regular retina checkups for many diabetic patients and reduce the burden on the limited number of retina experts by referring only those patients who require their attention. Over the last decade, numerous deep neural network-based algorithms have been proposed for ADRCs to distinguish the severity levels of DR. However, it has not been investigated whether DNN-based ADRCs consider the same criteria as human retina professionals (HRPs), i.e., whether they follow the same grading scale when making decisions about the severity level of DR, which may put the reliability of ADRCs into question. In this study, we investigated this issue by experimenting on publicly available datasets using MobileNet-based ADRCs and analyzing the output of the ADRCs using two eXplainable artificial intelligence (XAI) techniques named Gradient-weighted Class Activation Map (Grad-CAM) and Integrated Gradients (IG).