DOI 10.1016/j.jpainsymman.2026.04.002
Chatbot Responses to Frequently Asked Questions About Cannabis and Its Use for Cancer Symptoms.
Аннотация
CONTEXT: Chatbots are increasingly used by the public, but their performance in answering questions about complex health topics, such as cannabis, is unknown.
OBJECTIVES
To evaluate responses of three popular chatbots regarding cannabis and its use for cancer-related symptoms.
METHODS
We asked ChatGPT, Google Gemini, and Microsoft Co-Pilot to answer questions about cannabis derived from the Centers for Disease Control website and American Society of Clinical Oncology guidelines regarding cannabis. Responses were collected on February 6, 2025. Six physicians with expertise in this field scored responses for accuracy and comprehensiveness (0-10 scale). Reliability of references was scored separately (0-10 scale). Readability was assessed using Flesch-Kincaid Grade Level, Flesch Reading Ease scores.
RESULTS
Mean accuracy scores (SD) for ChatGPT, Gemini, and Co-Pilot were 9.0 (1.8), 8.8 (2.3), and 8.3 (2.3), respectively. Co-Pilot significantly underperformed in accuracy compared to ChatGPT (mean difference -0.62, 95% CI: -1.11, 0.14; P = 0.008). Mean comprehensiveness scores (SD) for ChatGPT, Gemini, and Co-Pilot were 8.1 (2.2), 8.5 (2.2), and 7.2 (2.4), respectively. ChatGPT and Gemini performed better than Co-Pilot in comprehensiveness (mean difference Co-Pilot vs. ChatGPT: -0.88 [95% CI: 1.34, -0.42; P < 0.001]; mean difference Co-Pilot vs. Gemini: -1.28 [95% CI: -1.74, -0.82; P < 0.001]). Inaccurate or misleading statements regarding cannabis formulations and symptom benefits were identified, with missing information on adverse effects and drug interactions. Gemini had the lowest reliability (4.1). Readability among all chatbots was poor.
CONCLUSION
Despite overall high accuracy and comprehensiveness scores, chatbots made some misleading, inaccurate statements or missed information. For now, their answers should be interpreted with caution.
Предметные рубрики MeSH
Подготовка и перевод публикации
Обработка статьи · этапы, ошибки и управление
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Тип публикации указан по данным источника. Уровень доказательности автоматически не присваивается.