Evaluation of the Quality, Reliability, and Readability of ChatGPT-4 Responses on Exercise and Rehabilitation Strategies for Adolescent Myositis.

PURPOSE: The study aims to evaluate the ability of ChatGPT-4 to generate reliable and accurate responses concerning exercise and rehabilitation strategies for adolescent patients with myositis.

METHODS

Seventy frequently asked questions related to exercise and rehabilitation in adolescent myositis were developed and classified into 7 thematic categories. Information reliability was assessed using the modified DISCERN (mDISCERN) tool, quality was evaluated with the Global Quality Scale (GQS), accuracy was measured using a five-point Likert scale, and readability was assessed with the Flesch Reading Ease scale. Two independent physiotherapists with expertise in rheumatologic rehabilitation independently evaluated the responses.

RESULTS

The mDISCERN scale scores ranged from 3 to 3.67, with a mean of 3.36. The GQS scores varied from 3.46 to 5.0, with a mean of 3.86. Accuracy scale scores ranged from 3.9 to 5.0, with a mean of 4.26. The Flesch Reading Ease scores ranged from 42.92 to 55.61, with a mean of 47.79. The intraclass correlation coefficients for the mDISCERN, GQS, and accuracy scales were 0.773, 0.712, and 0.710, respectively.

DISCUSSION: This study emphasized that ChatGPT-4 responses regarding adolescent myositis are generally accurate, with moderate to good reliability. However, the high reading level, requiring college-level education, may limit accessibility. These limitations underscore the need for healthcare professionals to supervise exercise planning and for further development of domain-specific AI models with enhanced reliability and readability.