MedElevator Войти в исследования
← Поиск публикаций

DOI 10.1136/jitc-2023-008599

Медицинская библиотека · Journal for immunotherapy of cancer · 2024

Use of artificial intelligence chatbots in clinical management of immune-related adverse events.

Burnette Hannah, Pabani Aliyah, von Itzstein Mitchell S, Switzer Benjamin, Fan Run, Ye Fei, Puzanov Igor, Naidoo Jarushka, Ascierto Paolo A, Gerber David E, Ernstoff Marc S, Johnson Douglas B

СтатьяПеревод пока не подготовлен

Аннотация

BACKGROUND

Artificial intelligence (AI) chatbots have become a major source of general and medical information, though their accuracy and completeness are still being assessed. Their utility to answer questions surrounding immune-related adverse events (irAEs), common and potentially dangerous toxicities from cancer immunotherapy, are not well defined.

METHODS

We developed 50 distinct questions with answers in available guidelines surrounding 10 irAE categories and queried two AI chatbots (ChatGPT and Bard), along with an additional 20 patient-specific scenarios. Experts in irAE management scored answers for accuracy and completion using a Likert scale ranging from 1 (least accurate/complete) to 4 (most accurate/complete). Answers across categories and across engines were compared.

RESULTS

Overall, both engines scored highly for accuracy (mean scores for ChatGPT and Bard were 3.87 vs 3.5, p<0.01) and completeness (3.83 vs 3.46, p<0.01). Scores of 1-2 (completely or mostly inaccurate or incomplete) were particularly rare for ChatGPT (6/800 answer-ratings, 0.75%). Of the 50 questions, all eight physician raters gave ChatGPT a rating of 4 (fully accurate or complete) for 22 questions (for accuracy) and 16 questions (for completeness). In the 20 patient scenarios, the average accuracy score was 3.725 (median 4) and the average completeness was 3.61 (median 4).

CONCLUSIONS

AI chatbots provided largely accurate and complete information regarding irAEs, and wildly inaccurate information ("hallucinations") was uncommon. However, until accuracy and completeness increases further, appropriate guidelines remain the gold standard to follow.

Для цитирования: Burnette Hannah, Pabani Aliyah, von Itzstein Mitchell S, Switzer Benjamin, Fan Run, Ye Fei, Puzanov Igor, Naidoo Jarushka, Ascierto Paolo A, Gerber David E, Ernstoff Marc S, Johnson Douglas B. Use of artificial intelligence chatbots in clinical management of immune-related adverse events.. Journal for immunotherapy of cancer. 2024. DOI: 10.1136/jitc-2023-008599.

Предметные рубрики MeSH

Подготовка и перевод публикации
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…

История · было → стало

Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.

Тип публикации указан по данным источника. Уровень доказательности автоматически не присваивается.