Artificial Intelligence Chatbots and Their Responses to Most Searched Spanish Cancer Questions.

BACKGROUND

Artificial intelligence (AI) chatbots perform well in answering English cancer questions. For Spanish, their performance is unknown and may differ by free vs. paywall versions.

METHODS

We evaluated the quality (range: 1-5 points), actionability (range: 0-100%), and readability (range: 1-13 grades) of six popular AI chatbots in responding to the 15 most searched Spanish questions regarding breast, prostate, and colon cancer.

RESULTS

The quality of overall AI chatbot responses was good (mean [95% CI]: 3.5 [3.4-3.6] points), while the actionability was low (mean [95% CI]: 35.6% [30.8%-40.3%]). The readability was high-school-level (mean [95% CI]: 9.2 [8.8-9.6] grades), not concordant with the American Medical Association recommendation (≤ 6th grade). The quality, actionability, and readability did not differ by free and paywall versions (p > 0.05).

CONCLUSION

Our findings suggested AI chatbots may generate good-quality responses to Spanish cancer questions, regardless of free or paywall versions. However, further improvement in actionability and readability is needed to benefit Spanish-speaking patients.