Evaluating the accuracy and readability of ChatGPT-4 responses about cardiac rehabilitation for heart failure patients.

BACKGROUND

This study aimed to evaluate the accuracy and readability of ChatGPT-4 responses related to cardiac rehabilitation (CR) for patients with heart failure (HF), with the objective of assessing its potential as a patient education tool.

METHODS

The study involved 16 open-ended questions related to CR, developed by two specialists (one cardiologist and one physical medicine and rehabilitation specialist). These questions were submitted to ChatGPT-4, and its responses were evaluated for accuracy and readability. Accuracy was assessed using a 6-point Likert scale, while readability was analysed using the Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), Coleman-Liau Index (CLI), and Gunning Fog Index (GFI). Inter-evaluator reliability was assessed by the intraclass correlation coefficient (ICC).

RESULTS

The mean accuracy score of ChatGPT-4 responses was high (5.25 ± 0.77 and 5.38 ± 0.62 for two raters), with 81.25% of responses rated 5 or above. The readability analysis revealed a median FRE of 59.5, indicating moderate readability, with FKGL at 7.1 and CLI at 11.2. The ICC between the two evaluators was 0.854, indicating good agreement.

CONCLUSION

ChatGPT-4 provided accurate and reliable information on CR for HF patients. Although the readability was slightly above the ideal level, its overall performance suggests potential as a supportive tool in patient education. Further improvements in language simplicity are needed to optimise its usability.