Cureus · 2025
Can Large Language Models Draft Safe, Reliable Patient Leaflets on Driving After Stroke in Comparison to the Stroke Association Leaflet?
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Suleman Mohammed, Massraf Bnar, Licenik Radim
Аннотация
BACKGROUND: Readable and reliable patient information leaflets (PILs) are essential to ensuring patients understand post-stroke driving regulations. With the rapid advancement of large language models (LLMs), there is increasing interest in whether AI-generated materials can achieve quality comparable to expert-authored leaflets such as those produced by the Stroke Association.
OBJECTIVE: This study aimed to evaluate the readability, reliability, actionability, and factual accuracy of eight "Driving After Stroke" leaflets, seven generated by leading LLMs and one gold-standard version from the Stroke Association, using validated appraisal tools.
METHODS: Eight PILs were assessed: the official Stroke Association leaflet and seven AI-generated versions from ChatGPT-5 (Thinking, OpenAI, San Francisco, CA), ChatGPT-5o (Standard, OpenAI), Microsoft Copilot (Microsoft Corp., Redmond, WA), Claude Sonnet 4 (Anthropic, San Francisco, CA), Gemini 2.5 Flash (Google, Mountain View, CA), Perplexity AI (free, Perplexity AI, Inc., San Francisco, CA), and DeepSeek v3.1 (DeepSeek, Hangzhou, China). Each leaflet was converted to plain text for uniform analysis. Readability (Flesch-Kincaid Grade (FKG), Flesch Reading Ease (FRE), Automated Readability Index (ARI), and Gunning Fog Index (GFI)), reliability (DISCERN), understandability and actionability (Patient Education Materials Assessment Tool for Print Materials (PEMAT-P; understandability (U)/actionability (A)), misinformation (0-6; lower = better), and referencing quality (0-2) were scored and synthesised into a weighted composite, adapted from a prior ophthalmology study of AI-generated patient leaflets.
RESULTS: The Stroke Association leaflet achieved the highest composite score (72.8%), followed by DeepSeek v3.1 (68.3%) and Microsoft Copilot (66.4%). ChatGPT-5 (Thinking) ranked closely behind (66.3%), while Gemini 2.5 Flash, ChatGPT-5o (Standard), Claude Sonnet 4, and Perplexity AI (free) scored lower (range 29.8-59.1%). Although no AI model exceeded the gold standard, DeepSeek produced near-comparable readability and structure. Across all leaflets, readability remained within a secondary-school band (median FKG 8.5), with AI-generated texts generally clearer but less well-referenced. The Stroke Association leaflet achieved perfect factual accuracy and referencing, whereas Microsoft Copilot and DeepSeek demonstrated occasional omissions in seizure and transient ischemic attack (TIA) guidance.
CONCLUSION: Expert-authored materials remain the benchmark for reliability, factual precision, and source transparency. However, newer LLMs such as DeepSeek v3.1 and Microsoft Copilot generated patient leaflets that approached professional standards in readability and usability. These findings support a hybrid approach, AI-assisted drafting with clinician and legal oversight is the most efficient and safe model for future patient education.
OBJECTIVE: This study aimed to evaluate the readability, reliability, actionability, and factual accuracy of eight "Driving After Stroke" leaflets, seven generated by leading LLMs and one gold-standard version from the Stroke Association, using validated appraisal tools.
METHODS: Eight PILs were assessed: the official Stroke Association leaflet and seven AI-generated versions from ChatGPT-5 (Thinking, OpenAI, San Francisco, CA), ChatGPT-5o (Standard, OpenAI), Microsoft Copilot (Microsoft Corp., Redmond, WA), Claude Sonnet 4 (Anthropic, San Francisco, CA), Gemini 2.5 Flash (Google, Mountain View, CA), Perplexity AI (free, Perplexity AI, Inc., San Francisco, CA), and DeepSeek v3.1 (DeepSeek, Hangzhou, China). Each leaflet was converted to plain text for uniform analysis. Readability (Flesch-Kincaid Grade (FKG), Flesch Reading Ease (FRE), Automated Readability Index (ARI), and Gunning Fog Index (GFI)), reliability (DISCERN), understandability and actionability (Patient Education Materials Assessment Tool for Print Materials (PEMAT-P; understandability (U)/actionability (A)), misinformation (0-6; lower = better), and referencing quality (0-2) were scored and synthesised into a weighted composite, adapted from a prior ophthalmology study of AI-generated patient leaflets.
RESULTS: The Stroke Association leaflet achieved the highest composite score (72.8%), followed by DeepSeek v3.1 (68.3%) and Microsoft Copilot (66.4%). ChatGPT-5 (Thinking) ranked closely behind (66.3%), while Gemini 2.5 Flash, ChatGPT-5o (Standard), Claude Sonnet 4, and Perplexity AI (free) scored lower (range 29.8-59.1%). Although no AI model exceeded the gold standard, DeepSeek produced near-comparable readability and structure. Across all leaflets, readability remained within a secondary-school band (median FKG 8.5), with AI-generated texts generally clearer but less well-referenced. The Stroke Association leaflet achieved perfect factual accuracy and referencing, whereas Microsoft Copilot and DeepSeek demonstrated occasional omissions in seizure and transient ischemic attack (TIA) guidance.
CONCLUSION: Expert-authored materials remain the benchmark for reliability, factual precision, and source transparency. However, newer LLMs such as DeepSeek v3.1 and Microsoft Copilot generated patient leaflets that approached professional standards in readability and usability. These findings support a hybrid approach, AI-assisted drafting with clinician and legal oversight is the most efficient and safe model for future patient education.