Child's nervous system : ChNS : official journal of the International Society for Pediatric Neurosurgery · 2026
Patient and physician perspectives on large language model generated responses about brain aneurysm.
СтатьяПоддержка исследования: вне правительства СШАНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Choi Joon Hyeok, Ruppert-Gomez Marcella, Shanahan Liam M, Staffa Steven J, Orbach Darren B, Smith Edward R, Oushy Soliman, Lehman Laura, Larson Anna M, Rangwala Shivani D, Buckley Christine, Hilton Grace, See Alfred P
Аннотация
PURPOSE: Large language models (LLMs) are becoming increasingly popular in medicine and neurosurgery. Because LLMs are not trained in specific subspecialties or diagnoses, a better understanding of the implications, effectiveness, and use of LLMs by users and clinicians is necessary. To better understand LLM's effectiveness in neurosurgery and aneurysms, we compared community and physician feedback on ChatGPT 4o and Gemini 1.5 Flash responses to frequently asked questions regarding brain aneurysms.
METHODS: External surveys were made available on the Brain Aneurysm Foundation page for patients and families to complete and internal surveys were distributed and completed by physicians in the department of neurosurgery at Boston Children's Hospital.
RESULTS: In the community survey assessing response usefulness and helpfulness, ChatGPT and Gemini provided different response quality despite similar AI sentiment. Clarity of procedure explanation (p = 0.04), discussion of alternative procedures (p = 0.02), and discussion of procedure risks (p = 0.01) were different. The physician survey, assessing response accuracy, safety, and helpfulness, also found differences in multiple domains. Importantly, differences were found in consistency with current medical knowledge and practice guidelines (p = 0.001), omittance of key points (p < 0.001), and amount of clinically relevant detail included (p < 0.001).
CONCLUSION: LLMs had variable performance across several key domains, consistent with previous research. Despite the apparent advantages of ChatGPT, physician feedback highlighted the continued need for information oversight. Interestingly, community participants consistently found LLM responses to be better than physician ones, while physicians found LLM responses to be similar or somewhat worse than the one they would have provided.
METHODS: External surveys were made available on the Brain Aneurysm Foundation page for patients and families to complete and internal surveys were distributed and completed by physicians in the department of neurosurgery at Boston Children's Hospital.
RESULTS: In the community survey assessing response usefulness and helpfulness, ChatGPT and Gemini provided different response quality despite similar AI sentiment. Clarity of procedure explanation (p = 0.04), discussion of alternative procedures (p = 0.02), and discussion of procedure risks (p = 0.01) were different. The physician survey, assessing response accuracy, safety, and helpfulness, also found differences in multiple domains. Importantly, differences were found in consistency with current medical knowledge and practice guidelines (p = 0.001), omittance of key points (p < 0.001), and amount of clinically relevant detail included (p < 0.001).
CONCLUSION: LLMs had variable performance across several key domains, consistent with previous research. Despite the apparent advantages of ChatGPT, physician feedback highlighted the continued need for information oversight. Interestingly, community participants consistently found LLM responses to be better than physician ones, while physicians found LLM responses to be similar or somewhat worse than the one they would have provided.