Cureus · 2026
Comparative Evaluation of Gemini 3.0- and ChatGPT 5.0-Generated Regional Language Informed Consent Forms in Ophthalmology: A Dual-Rater Study in Hindi and Kannada.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Das Deepsekhar, Shetti Bahubali, Antalmarad Venkatesh, Choudhary Anshum, Thodupunoori Kathyayini G, Grover Sumit, Narayan Atindra
Аннотация
PURPOSE: To evaluate the accuracy, linguistic quality, clinical completeness, and real-world applicability of informed consent forms generated in Indian regional languages (Hindi and Kannada) by two large language model-based chatbots for common ophthalmic surgical procedures.
METHODS: In this comparative, blinded observational study, two chatbots (Gemini 3.0 (Google, California, USA) and ChatGPT 5.0 (OpenAI, California, USA)) were prompted to generate informed consent documents in Hindi and Kannada for five ophthalmic scenarios: cataract surgery, traumatic corneal perforation repair, therapeutic penetrating keratoplasty, orbitotomy, and squint surgery. Outputs were independently assessed by four ophthalmologist raters (two for each language) using a 10-point scoring system based on correctness, completeness, language and readability, clinical relevance, and real-world applicability. Descriptive statistics were calculated. Paired t-tests were used to compare chatbot performance, effect sizes (Cohen's d) were estimated, and inter-rater reliability was assessed using intraclass correlation coefficients (ICCs).
RESULTS: Across both languages, Gemini 3.0 demonstrated more consistent performance and higher combined mean scores. In the Hindi cohort, combined mean scores were comparable between Gemini 3.0 (7.85) and ChatGPT 5.0 (8.00), with significant rater-dependent preference variability. In contrast, in the Kannada cohort, Gemini 3.0 significantly outperformed ChatGPT 5.0 (8.8 vs 7.7, p<0.01), with large to extremely large effect sizes (Cohen's d: 1.23-3.8). Inter-rater reliability was moderate to good for Gemini 3.0 (ICC: 0.62-0.71) and lower for ChatGPT 5.0 (ICC: 0.38-0.59). ChatGPT 5.0 exhibited frequent grammatical and terminological inaccuracies, particularly in Kannada, affecting clinical usability.
CONCLUSION: Large language models can generate clinically usable informed consent forms in Indian regional languages; however, performance varies significantly between models. Gemini 3.0 demonstrated superior linguistic accuracy, consistency, and clinical suitability. Language-specific validation and mandatory human oversight are essential before clinical implementation.
METHODS: In this comparative, blinded observational study, two chatbots (Gemini 3.0 (Google, California, USA) and ChatGPT 5.0 (OpenAI, California, USA)) were prompted to generate informed consent documents in Hindi and Kannada for five ophthalmic scenarios: cataract surgery, traumatic corneal perforation repair, therapeutic penetrating keratoplasty, orbitotomy, and squint surgery. Outputs were independently assessed by four ophthalmologist raters (two for each language) using a 10-point scoring system based on correctness, completeness, language and readability, clinical relevance, and real-world applicability. Descriptive statistics were calculated. Paired t-tests were used to compare chatbot performance, effect sizes (Cohen's d) were estimated, and inter-rater reliability was assessed using intraclass correlation coefficients (ICCs).
RESULTS: Across both languages, Gemini 3.0 demonstrated more consistent performance and higher combined mean scores. In the Hindi cohort, combined mean scores were comparable between Gemini 3.0 (7.85) and ChatGPT 5.0 (8.00), with significant rater-dependent preference variability. In contrast, in the Kannada cohort, Gemini 3.0 significantly outperformed ChatGPT 5.0 (8.8 vs 7.7, p<0.01), with large to extremely large effect sizes (Cohen's d: 1.23-3.8). Inter-rater reliability was moderate to good for Gemini 3.0 (ICC: 0.62-0.71) and lower for ChatGPT 5.0 (ICC: 0.38-0.59). ChatGPT 5.0 exhibited frequent grammatical and terminological inaccuracies, particularly in Kannada, affecting clinical usability.
CONCLUSION: Large language models can generate clinically usable informed consent forms in Indian regional languages; however, performance varies significantly between models. Gemini 3.0 demonstrated superior linguistic accuracy, consistency, and clinical suitability. Language-specific validation and mandatory human oversight are essential before clinical implementation.