Ocular immunology and inflammation · 2026
Exploring Artificial Intelligence's Role in Citation Generation for Ocular Inflammation and Uveal Diseases Research: A Comparative Evaluation Across Four Models.
СтатьяСравнительное исследованиеНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Civelekler Mustafa, Citirik Mehmet
Аннотация
PURPOSE: This study evaluated four artificial intelligence (AI) models-ChatGPT, Copilot, DeepSeek, and Gemini-for their ability to generate PubMed citations related to ocular inflammation and uveal disease. The aim was to assess their performance in a specialized clinical context and determine whether these tools can support accurate academic referencing.
METHODS: Thirty-five clinical paragraphs from The Review of Ophthalmology (4th edition) were provided to each model, which was instructed to generate AMA 11-style PubMed citations. Outputs were examined for accuracy, DOI matching, and clinical relevance. Expert reviewers classified each citation as Fully Cited, Partially Cited, or Not Cited. Statistical differences among the models were assessed using ANOVA with post hoc analysis.
RESULTS: DeepSeek, a domain-specific model, outperformed other models with an accuracy of 65.7% (p < 0.001). Copilot and ChatGPT achieved moderate accuracy rates of 42.9% and 37.1% (p = 0.042), respectively, whereas Gemini performed the worst, with an accuracy of 5.7% (p < 0.001). These results were statistically significant, as confirmed by ANOVA and post hoc analysis. Additional errors included incorrect citations, DOI mismatches, and incomplete reference lists. Expert validation also showed that DeepSeek produced the highest number of fully accurate citations, while the remaining models generated more partial or uncited references.
CONCLUSION: AI tools can assist with citation generation, but their reliability varies significantly. Domain-specific systems perform better, yet inconsistencies such as partial citations and hallucinated details highlight the continued need for expert oversight. Accurate academic referencing still depends on combining AI-generated material with careful human review.
METHODS: Thirty-five clinical paragraphs from The Review of Ophthalmology (4th edition) were provided to each model, which was instructed to generate AMA 11-style PubMed citations. Outputs were examined for accuracy, DOI matching, and clinical relevance. Expert reviewers classified each citation as Fully Cited, Partially Cited, or Not Cited. Statistical differences among the models were assessed using ANOVA with post hoc analysis.
RESULTS: DeepSeek, a domain-specific model, outperformed other models with an accuracy of 65.7% (p < 0.001). Copilot and ChatGPT achieved moderate accuracy rates of 42.9% and 37.1% (p = 0.042), respectively, whereas Gemini performed the worst, with an accuracy of 5.7% (p < 0.001). These results were statistically significant, as confirmed by ANOVA and post hoc analysis. Additional errors included incorrect citations, DOI mismatches, and incomplete reference lists. Expert validation also showed that DeepSeek produced the highest number of fully accurate citations, while the remaining models generated more partial or uncited references.
CONCLUSION: AI tools can assist with citation generation, but their reliability varies significantly. Domain-specific systems perform better, yet inconsistencies such as partial citations and hallucinated details highlight the continued need for expert oversight. Accurate academic referencing still depends on combining AI-generated material with careful human review.