Assessing Readability and DISCERN Quality of Osteoporosis Education Materials Generated by ChatGPT and Deepseek for Diverse Health Literacy Levels: A Cross-Sectional Study.

BACKGROUND AND AIMS: Large language models (LLMs) are increasingly used for patient education, but their output quality across health literacy levels is unknown. This cross-sectional study compared ChatGPT-4o, ChatGPT-5, DeepSeek-V3.1, and DeepSeek-R2 in generating osteoporosis education materials tailored to low, moderate, and high health literacy.

METHODS

Six clinical domains were posed to each LLM, with prompts adapted for three literacy tiers per model (18 outputs per model). Outputs were aggregated into 12 composite texts (4 models × 3 tiers). Three blinded clinicians assessed information quality using DISCERN (0-80) and readability using Flesch-Kincaid (FKGL, lower = easier).

RESULTS

Mean DISCERN scores ranged 36-52/80 ("fair"); no output reached "excellent" (> 70/80). DeepSeek-V3.1 provided the highest treatment detail for high literacy and best low-literacy readability (FKGL 3.99). ChatGPT-5 performed most consistently across tiers. Median readability ranged from grade 4.8 to 10.2. All outputs lacked citations, publication dates, quantitative risk data, and uncertainty statements.

CONCLUSION

LLMs can rapidly generate readable osteoporosis education, but current outputs require supplementation with references, risk statistics, and updated transparency before clinical use.