Comparative Analysis of Readability Levels of Hand Surgery Patient Information on the ASSH Website and Artificial Intelligence.

BACKGROUND

The American Medical Association (AMA) recommends patient education materials (PEMs) be written at a sixth-grade level to ensure accessibility. This study aims to compare hand surgery topics from the American Society for Surgery of the Hand (ASSH) website and artificial intelligence (AI)-generated PEMs to evaluate their alignment with the AMA's recommendations.

METHODS

Ninety-eight hand surgery topics from the ASSH Hand Care "The Upper Extremity Expert" website were analyzed. Artificial intelligence (AI) platforms (Google Gemini 2.0 Flash and ChatGPT-4) were asked to: (1) create original PEMs based on ASSH website topics; (2) rewrite the ASSH topics at a sixth-grade reading level; and (3) create original PEMs over the ASSH topics at a sixth-grade reading level. Readability was assessed using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) scores.

RESULTS

The ASSH website and AI-generated PEMs initially exceeded the sixth-grade readability level. Original ASSH PEMs ranged from a readability of 10th to 12th-grade (FRE 57.5 [6.6]; FKGL 9.4 [1.2]). Google Gemini and ChatGPT's initial PEMs were at the college level (Google Gemini: FRE 49.7 [7.5]; FKGL 9.6 [1.1] vs ChatGPT: FRE 47.4 [9.1]; FKGL 10.4 [1.4]). Artificial intelligence-rewritten ASSH content had improved readability, and AI-generated PEMs at a sixth-grade readability had lower word counts, higher FRE, and lower FKGL than original ASSH PEMs (all P < .001).

CONCLUSIONS

Artificial intelligence platforms can be used as a tool to create hand surgery PEMs at the recommended sixth-grade reading level. Future efforts are needed to ensure the accuracy of AI-generated PEMs and accessibility to patients of varying health literacy for informed decision-making.