Comparing the readability of carpal tunnel patient information between American academy of orthopedic surgeons' orthoinfo and artificial intelligence.

INTRODUCTION

Patients have become increasingly reliant on the internet to seek health-related information (HRI). The newfound popularity of artificial intelligence (AI) search engines has created interest in their ability to provide HRI. This study aimed to quantify and compare the readability of carpal tunnel syndrome (CTS) HRI from the American Academy of Orthopaedic Surgeons OrthoInfo and AI search engines.

METHODS

Six prompts were developed using the OrthoInfo page on CTS. These prompts were entered to ChatGPT-4 and Google Gemini 2.0 Flash to generate AI responses. The readability of this information was calculated using the Flesch-Kincaid Reading Ease Index, Coleman-Liau Index, Flesch-Kincaid Grade Level, FORCAST Readability Formula, Gunning Fog index, and Simple Measure of Gobbledygook Index. Statistical testing was performed using the Kruskal-Wallis nonparametric One-Way Analysis of Variance test.

RESULTS

The mean grade level readability score across all platforms, questions, and testing metrics was 12.6. No significant differences were observed between the overall mean grade level readability scores of OrthoInfo, ChatGPT, and Gemini, nor were they observed for any specific prompt. The only significant differences were found using the Flesch-Kincaid Grade Level test, for which ChatGPT had the lowest scores.

CONCLUSION

The readability of carpal tunnel syndrome health-related information from OrthoInfo, ChatGPT, and Gemini is similar. Physicians should advise patients to continue using OrthoInfo as a primary source of carpal tunnel syndrome information, although artificial intelligence search engines are useful to supplement when patient concerns require more tailored responses. Notably, no text included in this study was at recommended reading level thresholds.