Contact lens & anterior eye : the journal of the British Contact Lens Association · 2025
Diagnostic accuracy in dry eye: Insights into clinical and artificial intelligence limitations: Limitations of diagnostic accuracy in dry eye.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Mejía-Salgado Germán, Rojas-Carabali William, Cifuentes-González Carlos, Bernal-Valencia María Andrea, Saboya-Galindo Paola, Soto-Ariño Jaime, Dumar-Kerguelen Valentina, Marroquín-Gómez Guillermo, Moreno-Pardo Martha Lucía, Tirado-Ángel Juliana, Galor Anat, de-la-Torre Alejandra
Аннотация
PURPOSE: To evaluate the agreement and performance of four large language models (LLMs)-ChatGPT-3.5, ChatGPT-4.0, Leny-ai, and MediSearch-in diagnosing and classifying Dry Eye Disease (DED), compared to clinician judgment and Dry Eye Workshop-II (DEWS-II) criteria.
METHODS: A standardized prompt incorporating retrospective clinical and symptomatic data from patients with suspected DED referred to a dry eye clinic was developed. LLMs were evaluated for diagnosis (DED vs. no DED) and classification (aqueous-deficient, evaporative, mixed-component). Agreement was assessed using Cohen's-kappa (Cκ) and Fleiss'-kappa (Fκ). Balanced accuracy, sensitivity, specificity, and F1 score were calculated.
RESULTS: Among 338 patients (78.6 % female, mean age 53.2 years), clinicians diagnosed DED in 300, and DEWS-II criteria identified 234. LLMs showed high agreement with clinicians for DED diagnosis (93 %-99 %, Cκ: 0.81-0.86). Subtype agreement was lower (aqueous-deficient: 0 %-18 %, evaporative: 4 %-80 %, mixed-component: 22 %-92 %; Fκ: -0.20 to -0.10). Diagnostic balanced accuracy was 48 %-56 %, with high sensitivity (93 %-99 %) but low specificity (0 %-16 %). Subtype balanced accuracy and F1 score ranged from 33 %-81 % 0 %-71 %, respectively. Compared to DEWS-II, agreement for DED diagnosis remained high (96 %-99 %) but with weaker Cκ (0.52-0.58). Subtype agreement was again low (aqueous-deficient: 0 %-20 %, evaporative: 9 %-68 %, mixed-component: 16 %-75 %; Fκ: -0.09 to -0.02). Diagnostic balanced accuracy was 49 %-56 %, sensitivity 97 %-99 %, and specificity 5 %-16 %. Subtype balanced accuracy ranged from 43 % to 56 %, F1 score 0-68.
CONCLUSION: LLMs showed strong agreement and high sensitivity for DED diagnosis but limited specificity and poor subtype classification, mirroring clinical challenges and highlighting risks of overdiagnosis.
METHODS: A standardized prompt incorporating retrospective clinical and symptomatic data from patients with suspected DED referred to a dry eye clinic was developed. LLMs were evaluated for diagnosis (DED vs. no DED) and classification (aqueous-deficient, evaporative, mixed-component). Agreement was assessed using Cohen's-kappa (Cκ) and Fleiss'-kappa (Fκ). Balanced accuracy, sensitivity, specificity, and F1 score were calculated.
RESULTS: Among 338 patients (78.6 % female, mean age 53.2 years), clinicians diagnosed DED in 300, and DEWS-II criteria identified 234. LLMs showed high agreement with clinicians for DED diagnosis (93 %-99 %, Cκ: 0.81-0.86). Subtype agreement was lower (aqueous-deficient: 0 %-18 %, evaporative: 4 %-80 %, mixed-component: 22 %-92 %; Fκ: -0.20 to -0.10). Diagnostic balanced accuracy was 48 %-56 %, with high sensitivity (93 %-99 %) but low specificity (0 %-16 %). Subtype balanced accuracy and F1 score ranged from 33 %-81 % 0 %-71 %, respectively. Compared to DEWS-II, agreement for DED diagnosis remained high (96 %-99 %) but with weaker Cκ (0.52-0.58). Subtype agreement was again low (aqueous-deficient: 0 %-20 %, evaporative: 9 %-68 %, mixed-component: 16 %-75 %; Fκ: -0.09 to -0.02). Diagnostic balanced accuracy was 49 %-56 %, sensitivity 97 %-99 %, and specificity 5 %-16 %. Subtype balanced accuracy ranged from 43 % to 56 %, F1 score 0-68.
CONCLUSION: LLMs showed strong agreement and high sensitivity for DED diagnosis but limited specificity and poor subtype classification, mirroring clinical challenges and highlighting risks of overdiagnosis.