Evaluation of the degree of agreement in the diagnosis of diabetic retinopathy between ophthalmologists and EyeArt®.

OBJECTIVE OR PURPOSE: To evaluate the diagnostic performance and agreement of the EyeArt® Artificial Intelligence (AI) system for detecting Diabetic Retinopathy (DR), comparing its results with ophthalmologists' assessments in a regional screening program.

DESIGN

Cross-sectional observational study.

SUBJECTS, PARTICIPANTS, AND/OR CONTROLS: A total of 498 diabetic patients aged 18 years or older were enrolled between June and September 2023 through the Retisalud DR screening program in the Canary Islands. No separate control group was included.

METHODS

All participants underwent non-mydriatic fundus photography using the TRC-NW400 camera. Retinal images were analyzed by the EyeArt® AI system (version 2.1.0), and results were compared with assessments by ophthalmologists based on the International Clinical Diabetic Retinopathy scale (ICDR). Agreement was quantified using Cohen's kappa coefficient. Additionally, mixed-effects logistic regression was used to explore associations between DR and clinical risk factors.

MAIN OUTCOME MEASURES: Sensitivity, specificity, and agreement (Cohen's kappa) of the AI system compared to clinical diagnosis; predictors of DR such as age, diabetes duration, presence of Diabetic Macular Edema (DME), and central retinal thickness (CRT-OCT).

RESULTS

The EyeArt® system achieved a binocular sensitivity of 100% (95% CI: 98.1-100) and a specificity of 93.5% (95% CI: 90.2-96.0). Agreement with ophthalmologist grading was excellent, with kappa values of 0.966 (right eye) and 0.978 (left eye). Younger age, longer diabetes duration, DME presence, and higher CRT were significantly associated with DR diagnosis.

CONCLUSIONS

The EyeArt® AI system showed excellent diagnostic accuracy and strong agreement with clinical evaluations in DR screening. Nonetheless, its tendency to overestimate DR severity indicates the need for further refinement of its grading algorithm. These findings support the potential integration of AI systems into large-scale DR screening programs, pending further validation.