Journal francais d'ophtalmologie · 2025
Comparative performance of ChatGPT o3-mini-high and DeepSeek-R1 in ophthalmology: An evaluation of diagnostic reasoning and case-based problem solving.
СтатьяСравнительное исследованиеНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Karataş G, Kırık F, Karataş M E, Çakır A, Özdemir H
Аннотация
PURPOSE: To compare the diagnostic reasoning and case-based problem-solving abilities of ChatGPT o3-mini-high and DeepSeek-R1 in ophthalmological cases with text-based questions.
METHODS: Fifty-five consecutive text-based case-solving questions from nine ophthalmology subspecialties were posed to two reasoning-capable LLMs, ChatGPT o3-mini-high and DeepSeek-R1. For each case, the multi-component diagnostic reasoning approach described by Elstein was applied. Overall diagnostic accuracy, diagnostic agreement between the models, reasoning competence, subspecialty-specific performance, and the tendency to request additional prompts were recorded. Two expert ophthalmologists then independently rated each model's diagnostic reasoning ability for all questions, using the Global Quality Score, a five-point scale (1=poor; 5=excellent).
RESULTS: ChatGPT o3-mini-high correctly answered 80% of the questions, whereas DeepSeek-R1 achieved a correct response rate of 54.5% (P<0.001), and Cohen's kappa coefficient was 0.462. ChatGPT o3-mini-high tended to request additional prompts for responses to fewer questions (2 vs. 12; P: 0.013). For both LLMs, the highest accuracy was observed in the retina/vitreous-related cases, while the lowest accuracy was noted in glaucoma-related cases. When Elstein's medical reasoning components were evaluated with the GQS, ChatGPT o3-mini-high achieved a median score of 4.5 (IQR 2.5-5.0), whereas DeepSeek-R1 achieved 2.5 (IQR 1.0-4.5) (P<0.001). The weighted kappa was 0.407, indicating moderate agreement between the two models.
CONCLUSION: This study provides evidence that ChatGPT o3-mini-high demonstrates superior diagnostic accuracy and reasoning capabilities in the analysis of ophthalmologic cases compared to DeepSeek-R1.
METHODS: Fifty-five consecutive text-based case-solving questions from nine ophthalmology subspecialties were posed to two reasoning-capable LLMs, ChatGPT o3-mini-high and DeepSeek-R1. For each case, the multi-component diagnostic reasoning approach described by Elstein was applied. Overall diagnostic accuracy, diagnostic agreement between the models, reasoning competence, subspecialty-specific performance, and the tendency to request additional prompts were recorded. Two expert ophthalmologists then independently rated each model's diagnostic reasoning ability for all questions, using the Global Quality Score, a five-point scale (1=poor; 5=excellent).
RESULTS: ChatGPT o3-mini-high correctly answered 80% of the questions, whereas DeepSeek-R1 achieved a correct response rate of 54.5% (P<0.001), and Cohen's kappa coefficient was 0.462. ChatGPT o3-mini-high tended to request additional prompts for responses to fewer questions (2 vs. 12; P: 0.013). For both LLMs, the highest accuracy was observed in the retina/vitreous-related cases, while the lowest accuracy was noted in glaucoma-related cases. When Elstein's medical reasoning components were evaluated with the GQS, ChatGPT o3-mini-high achieved a median score of 4.5 (IQR 2.5-5.0), whereas DeepSeek-R1 achieved 2.5 (IQR 1.0-4.5) (P<0.001). The weighted kappa was 0.407, indicating moderate agreement between the two models.
CONCLUSION: This study provides evidence that ChatGPT o3-mini-high demonstrates superior diagnostic accuracy and reasoning capabilities in the analysis of ophthalmologic cases compared to DeepSeek-R1.