Implementing Generative AI to Enhance Patient Education on Retinopathy of Prematurity.

PURPOSE: To evaluate the efficacy of large language models (LLMs) in generating patient education materials (PEMs) on retinopathy of prematurity (ROP).

METHODS

ChatGPT-3.5 (OpenAI), ChatGPT-4 (OpenAI), and Gemini (Google AI) were compared on three separate prompts. Prompt A requested that each LLM generate a novel PEM on ROP. Prompt B requested generated PEMs at the 6th-grade reading level using the validated Simple Measure of Gobbledygook (SMOG) readability formula. Prompt C requested LLMs improve the readability of existing, human-written PEMs to a 6th-grade reading level. PEMs inserted into Prompt C were sourced through a Google search of "retinopathy of prematurity." Each PEM was analyzed for readability (SMOG, Flesch-Kincaid Grade Level [FKGL]), quality (Patient Education Materials Assessment Tool [PEMAT], DISCERN), and accuracy (Likert Misinformation Scale).

RESULTS

LLM-generated PEMs were of high quality (median DISCERN = 4), understandable (PEMAT-U ≥ 70%), and accurate (Likert = 1). Prompt B generated more readable PEMs than Prompt A (P < .001). ChatGPT-4 and Gemini rewrote PEMs (Prompt C) from a baseline readability level (FKGL: 8.8 ± 1.9, SMOG: 8.6 ± 1.5) to the targeted 6th-grade reading level. Only ChatGPT-4 rewrites maintained high quality and reliability (median DISCERN = 4).

CONCLUSIONS

LLMs, particularly ChatGPT-4, can serve as strong supplementary tools to automate the process of generating readable and high-quality PEMs for parents on ROP.