PloS one · 2026
Comparative evaluation of ChatGPT and gemini responses to patient-oriented questions on breast cancer.
СтатьяСравнительное исследованиеНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Yilmaz Bozok Yeliz, Acar Nihan, Atahan Murat Kemal, Karaali Cem
Аннотация
BACKGROUND: Breast cancer, the most common malignancy among women, remains a major global public health concern. With the rapid growth of artificial intelligence-based language models, it is essential to evaluate their potential roles in patient education. This study compared ChatGPT and Gemini in responding to patient-oriented questions about breast cancer and its surgical treatment regarding scientific accuracy, clarity, and unnecessary detail.
METHODS: Forty frequently asked questions were collected from Turkish online sources. Both models were queried under identical conditions on August 13, 2025, and the responses were anonymized for blinded evaluation. Four general surgeons experienced in breast surgery independently assessed each response using a five-point Likert scale across three domains: scientific accuracy, clarity, and unnecessary detail.
RESULTS: Response lengths were recorded and compared. ChatGPT achieved significantly higher median scores than Gemini in scientific accuracy [4.75 (3.50-5.00) vs. 4.25 (3.50-5.00); p < 0.001], clarity [4.75 (3.50-5.00) vs. 4.25 (3.25-5.00); p = 0.005], and unnecessary detail [5.00 (5.00-5.00) vs. 4.50 (3.50-5.00); p < 0.001]. The overall median score was 4.83 (4.08-5.00) for ChatGPT and 4.33 (3.50-4.92) for Gemini (p < 0.001). Gemini's responses were significantly longer (244 ± 84 vs. 170 ± 47 words; p < 0.001).
CONCLUSION: Both models demonstrated acceptable overall performance. However, ChatGPT's higher scores and shorter, more focused responses indicate that it may be a more efficient tool for addressing breast cancer-related patient questions. In their current form, these models should not be used for diagnostic or clinical decision-making purposes. AI-generated health information should be used only under expert supervision and for educational or supportive purposes.
METHODS: Forty frequently asked questions were collected from Turkish online sources. Both models were queried under identical conditions on August 13, 2025, and the responses were anonymized for blinded evaluation. Four general surgeons experienced in breast surgery independently assessed each response using a five-point Likert scale across three domains: scientific accuracy, clarity, and unnecessary detail.
RESULTS: Response lengths were recorded and compared. ChatGPT achieved significantly higher median scores than Gemini in scientific accuracy [4.75 (3.50-5.00) vs. 4.25 (3.50-5.00); p < 0.001], clarity [4.75 (3.50-5.00) vs. 4.25 (3.25-5.00); p = 0.005], and unnecessary detail [5.00 (5.00-5.00) vs. 4.50 (3.50-5.00); p < 0.001]. The overall median score was 4.83 (4.08-5.00) for ChatGPT and 4.33 (3.50-4.92) for Gemini (p < 0.001). Gemini's responses were significantly longer (244 ± 84 vs. 170 ± 47 words; p < 0.001).
CONCLUSION: Both models demonstrated acceptable overall performance. However, ChatGPT's higher scores and shorter, more focused responses indicate that it may be a more efficient tool for addressing breast cancer-related patient questions. In their current form, these models should not be used for diagnostic or clinical decision-making purposes. AI-generated health information should be used only under expert supervision and for educational or supportive purposes.