Digital journal of ophthalmology : DJO · 2025
Accuracy of an artificial intelligence chatbot in identifying congenital glaucoma from other ocular etiologies.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Cardakli Nur, Wang Bo, Doyle Jefferson J, Kraus Courtney L
Аннотация
PURPOSE: To evaluate the accuracy and appropriateness of responses provided by ChatGPT in identifying congenital glaucoma from a series of written and image-based prompts.
METHODS: A series of questions regarding common signs and symptoms of congenital glaucoma were developed and queried to ChatGPT-3.5 and ChatGPT-4.0, and a set of publicly available images of patients with congenital glaucoma were queried to the image search function of ChatGPT-4.0. Outputs were graded by three pediatric ophthalmologists with expertise in congenital glaucoma. Completeness of response, accuracy, potential for harm, and concern for glaucoma were assessed by each reviewer.
RESULTS: A higher proportion of prompt responses from ChatGPT-4.0 were graded to be acceptable/appropriate than from ChatGPT-3.5 (22/33 vs 9/33 [P = 0.001]) among text-based queries. A higher proportion of ChatGPT-4.0 responses were felt to raise appropriate concern for congenital glaucoma (8/11 vs 2/11 [P = 0.03]) and a lower proportion of responses had incorrect or inappropriate information of major clinical significance (0/33 vs 6/33 [P = 0.02]) than ChatGPT-3.5 responses. There was no significant difference in the proportion of responses from ChatGPT-3.5 and ChatGPT-4.0 that were deemed to have potential likelihood of harm (P = 0.17). Among clinical images queried to ChatGPT-4.0, responses to two of three images were universally felt to be unacceptable with a major amount of incorrect or inappropriate clinical information and high/definitive likelihood of harm. Among readability indices, the SMOG Index score showed more difficult readability scores for ChatGPT-4.0 than for ChatGPT-3.5 (14.8 ± 1.2 vs 14.0 ± 1.4 [P = 0.009]).
CONCLUSIONS: Despite superior performance from ChatGPT-4.0 compared with ChatGPT-3.5 in raising concern for congenital glaucoma and appropriateness of responses from text-based prompts, it performed poorly in recognizing clinical images of congenital glaucoma.
METHODS: A series of questions regarding common signs and symptoms of congenital glaucoma were developed and queried to ChatGPT-3.5 and ChatGPT-4.0, and a set of publicly available images of patients with congenital glaucoma were queried to the image search function of ChatGPT-4.0. Outputs were graded by three pediatric ophthalmologists with expertise in congenital glaucoma. Completeness of response, accuracy, potential for harm, and concern for glaucoma were assessed by each reviewer.
RESULTS: A higher proportion of prompt responses from ChatGPT-4.0 were graded to be acceptable/appropriate than from ChatGPT-3.5 (22/33 vs 9/33 [P = 0.001]) among text-based queries. A higher proportion of ChatGPT-4.0 responses were felt to raise appropriate concern for congenital glaucoma (8/11 vs 2/11 [P = 0.03]) and a lower proportion of responses had incorrect or inappropriate information of major clinical significance (0/33 vs 6/33 [P = 0.02]) than ChatGPT-3.5 responses. There was no significant difference in the proportion of responses from ChatGPT-3.5 and ChatGPT-4.0 that were deemed to have potential likelihood of harm (P = 0.17). Among clinical images queried to ChatGPT-4.0, responses to two of three images were universally felt to be unacceptable with a major amount of incorrect or inappropriate clinical information and high/definitive likelihood of harm. Among readability indices, the SMOG Index score showed more difficult readability scores for ChatGPT-4.0 than for ChatGPT-3.5 (14.8 ± 1.2 vs 14.0 ± 1.4 [P = 0.009]).
CONCLUSIONS: Despite superior performance from ChatGPT-4.0 compared with ChatGPT-3.5 in raising concern for congenital glaucoma and appropriateness of responses from text-based prompts, it performed poorly in recognizing clinical images of congenital glaucoma.