Assessing the performance of chat generative pretrained transformer (ChatGPT) in answering chronic kidney disease-related questions.

BACKGROUND

Chatbots produced by artificial intelligence are frequently used in health information today. We aimed to investigate the reliability and reproducibility of the answers given by Chat Generative Pretrained Transformer (ChatGPT), one of the most used chatbots, to frequently asked questions related to chronic kidney failure.

METHODS

We reviewed frequently asked questions related to chronic kidney disease (CKD) from social media platforms and Internet. The questions were asked to ChatGPT, and the answers were scored from 1 to 4 by two experienced nephrologists.

RESULTS

Eighty-five frequently asked questions about chronic renal failure were examined and 60 of them were included in the study after exclusion criteria. Fifty-one (85%) of the questions received 1 point, 7 (11.7%) received 2 points and 2 (3.3%) received 3 points. The similarity rates of the answers to the repeated questions were between 80% and 100%.

CONCLUSION

ChatGPT has provided reliable responses with high reproducibility to inquiries related to CKD.