How AI responds to common HIV/AIDS questions: ChatGPT versus DeepSeek.

BACKGROUND

Artificial intelligence (AI) has garnered significant attention and, to some extent, has even replaced certain search engines as a popular channel for people across regions to access information. HIV/AIDS is a common topic among infectious diseases, and patients or high-risk groups frequently search for information online. Our study evaluated the accuracy of two AI models (ChatGPT-3.5 and DeepSeek-R1) in answering HIV/AIDS-related questions.

METHODS

This was a cross-sectional analytical study comparing two advanced large language models representing Eastern and Western cultural backgrounds, respectively, with a gold standard (an infectious disease specialist). We compiled 26 common questions related to HIV/AIDS and categorized them into four themes (basic knowledge, diagnosis, treatment, and prevention). Three infectious disease experts independently scored the AI models' responses on a 4-point scale for each answer. There was no statistically significant difference in the scores for HIV/AIDS-related questions between the two AI models: Z = -2.135, two-sided p-value = 0.451 (p > 0.05). DeepSeek-R1 demonstrated higher accuracy, with 53.85% of its responses rated as "Excellent," compared to 48.72% for ChatGPT-3.5; there was no significant difference in accuracy between the two AI models when answering HIV/AIDS-related questions (χ 2 = 0.429, df = 1, p = 0.513). Both AI models achieved high average composite scores (DeepSeek-R1 = 3.50, ChatGPT-3.5 = 3.44, out of a maximum of 4 points). The AI models performed exceptionally well across all domains, though they were relatively weaker in the "Treatment" domain.

CONCLUSION

Our study has identified the potential of AI models, particularly ChatGPT-3.5 and DeepSeek-R1, to provide accurate and comprehensive responses to HIV/AIDS-related questions. As free online tools, they can provide personalized, useful medical information to diverse populations across regions. However, medical information generated by AI models should still be used under the supervision and review of healthcare professionals.