Analysis of ChatGPT-4's performance on ophthalmology questions from the MIR exam.

PURPOSE: To evaluate the performance of ChatGPT in solving clinical scenarios in ophthalmology, specifically questions from the specialty exams for Resident Medical Interns (MIR).

DESIGN

Cross-sectional design for evaluating a diagnostic tool.

METHOD: Ophthalmology questions from the MIR exams from the 2010-2023 sessions were collected. The performance of ChatGPT in successfully answering the questions was calculated. The results were also compared with those obtained by ophthalmology professionals. Additionally, sensitivity, specificity, and positive and negative probability coefficients were calculated.

RESULTS

A total of 54 questions were collected, with those from the subspecialty "Retina" being the most frequent. ChatGPT's overall score was 90.2%, with a sensitivity of 92.59% and a specificity of 96.8%. The average concordance with the evaluators' answers was 86.41%. The agreement between the evaluators was 79.62%.

CONCLUSIONS

ChatGPT-4 is a useful tool for solving clinical scenarios and theoretical questions in ophthalmology. Proper use of the tool, supervised by professionals, can help optimize the care processes for ophthalmology patients.