Can Large Language Models Draft Safe, Reliable Patient Leaflets on Driving After Stroke in Comparison to the Stroke Association Leaflet?

BACKGROUND

Readable and reliable patient information leaflets (PILs) are essential to ensuring patients understand post-stroke driving regulations. With the rapid advancement of large language models (LLMs), there is increasing interest in whether AI-generated materials can achieve quality comparable to expert-authored leaflets such as those produced by the Stroke Association.

OBJECTIVE

This study aimed to evaluate the readability, reliability, actionability, and factual accuracy of eight "Driving After Stroke" leaflets, seven generated by leading LLMs and one gold-standard version from the Stroke Association, using validated appraisal tools.

METHODS

Eight PILs were assessed: the official Stroke Association leaflet and seven AI-generated versions from ChatGPT-5 (Thinking, OpenAI, San Francisco, CA), ChatGPT-5o (Standard, OpenAI), Microsoft Copilot (Microsoft Corp., Redmond, WA), Claude Sonnet 4 (Anthropic, San Francisco, CA), Gemini 2.5 Flash (Google, Mountain View, CA), Perplexity AI (free, Perplexity AI, Inc., San Francisco, CA), and DeepSeek v3.1 (DeepSeek, Hangzhou, China). Each leaflet was converted to plain text for uniform analysis. Readability (Flesch-Kincaid Grade (FKG), Flesch Reading Ease (FRE), Automated Readability Index (ARI), and Gunning Fog Index (GFI)), reliability (DISCERN), understandability and actionability (Patient Education Materials Assessment Tool for Print Materials (PEMAT-P; understandability (U)/actionability (A)), misinformation (0-6; lower = better), and referencing quality (0-2) were scored and synthesised into a weighted composite, adapted from a prior ophthalmology study of AI-generated patient leaflets.

RESULTS

The Stroke Association leaflet achieved the highest composite score (72.8%), followed by DeepSeek v3.1 (68.3%) and Microsoft Copilot (66.4%). ChatGPT-5 (Thinking) ranked closely behind (66.3%), while Gemini 2.5 Flash, ChatGPT-5o (Standard), Claude Sonnet 4, and Perplexity AI (free) scored lower (range 29.8-59.1%). Although no AI model exceeded the gold standard, DeepSeek produced near-comparable readability and structure. Across all leaflets, readability remained within a secondary-school band (median FKG 8.5), with AI-generated texts generally clearer but less well-referenced. The Stroke Association leaflet achieved perfect factual accuracy and referencing, whereas Microsoft Copilot and DeepSeek demonstrated occasional omissions in seizure and transient ischemic attack (TIA) guidance.

CONCLUSION

Expert-authored materials remain the benchmark for reliability, factual precision, and source transparency. However, newer LLMs such as DeepSeek v3.1 and Microsoft Copilot generated patient leaflets that approached professional standards in readability and usability. These findings support a hybrid approach, AI-assisted drafting with clinician and legal oversight is the most efficient and safe model for future patient education.