Journal of pain and symptom management · 2024
AI-Generated Content in Cancer Symptom Management: A Comparative Analysis Between ChatGPT and NCCN.
СтатьяСравнительное исследованиеНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Lazris David, Schenker Yael, Thomas Teresa Hagan
Аннотация
BACKGROUND: Artificial intelligence-driven tools, like ChatGPT, are prevalent sources for online health information. Limited research has explored the congruity between AI-generated content and professional treatment guidelines. This study seeks to compare recommendations for cancer-related symptoms generated from ChatGPT with guidelines from the National Comprehensive Cancer Network (NCCN).
INTERVENTION: We extracted treatment recommendations for nine symptoms from NCCN, separated into four full Supportive Care sections and five subsections of the Palliative Care webpage. We entered "How can I reduce my cancer-related [symptom]" into ChatGPT- 3.5 for these same symptoms and extracted its recommendations. A comparative content analysis focused on recommendations for medications, consultations, and non-pharmacological strategies. We compared word count and Flesch-Kincaid Grade Level (FKGL) readability for each NCCN and ChatGPT section.
OUTCOMES: The mean percent agreement between NCCN and ChatGPT recommendations was 37.3% (range 16.7%-81.8%). NCCN offered more specific medication recommendations. ChatGPT did recommend medications in the constipation and diarrhea sections that were not recommended by NCCN. Significant differences in word count (P=0.03) and FKGL (P<0.01) were found for NCCN Supportive Care webpages, with ChatGPT having lower word count and reading level. In the NCCN Palliative Care webpage subsections, there was no significant difference in word count (P=0.076), but FKGL was significantly lower with ChatGPT (P<0.01).
CONCLUSIONS/LESSONS LEARNED: While ChatGPT provides concise, accessible supportive care advice, discrepancies with guidelines raise concerns for patient-facing symptom management recommendations. Future research should consider how AI can be used in conjunction with evidence-based guidelines to support cancer patients' supportive care needs.
INTERVENTION: We extracted treatment recommendations for nine symptoms from NCCN, separated into four full Supportive Care sections and five subsections of the Palliative Care webpage. We entered "How can I reduce my cancer-related [symptom]" into ChatGPT- 3.5 for these same symptoms and extracted its recommendations. A comparative content analysis focused on recommendations for medications, consultations, and non-pharmacological strategies. We compared word count and Flesch-Kincaid Grade Level (FKGL) readability for each NCCN and ChatGPT section.
OUTCOMES: The mean percent agreement between NCCN and ChatGPT recommendations was 37.3% (range 16.7%-81.8%). NCCN offered more specific medication recommendations. ChatGPT did recommend medications in the constipation and diarrhea sections that were not recommended by NCCN. Significant differences in word count (P=0.03) and FKGL (P<0.01) were found for NCCN Supportive Care webpages, with ChatGPT having lower word count and reading level. In the NCCN Palliative Care webpage subsections, there was no significant difference in word count (P=0.076), but FKGL was significantly lower with ChatGPT (P<0.01).
CONCLUSIONS/LESSONS LEARNED: While ChatGPT provides concise, accessible supportive care advice, discrepancies with guidelines raise concerns for patient-facing symptom management recommendations. Future research should consider how AI can be used in conjunction with evidence-based guidelines to support cancer patients' supportive care needs.