International journal of gynecological cancer : official journal of the International Gynecological Cancer Society · 2026
Not all artificial intelligence large language models are created equal: heterogeneous performance in therapeutic decision-making for recurrent epithelial ovarian cancer.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Borella Fulvio, Fucina Stefano, Denti Pietro, Gallio Niccolò, Bertero Luca, Ditto Antonino, Ray-Coquard Isabelle, Chopin Nicolas, Ferraioli Domenico, Cosma Stefano
Аннотация
OBJECTIVE: To evaluate the concordance between therapeutic recommendations generated by 3 widely used consumer large language models (Gemini, Microsoft Copilot, and ChatGPT) and the decisions of a multidisciplinary tumor board in the management of the first recurrence of high-grade serous ovarian cancer. Secondary objectives were to characterize recurrent error patterns and to assess the safety of model-generated recommendations using a 3-tier taxonomy.
METHODS: A retrospective observational analysis was conducted on 44 consecutive patients with a first recurrence of high-grade serous ovarian cancer treated at a tertiary referral center. For each case, an identical, anonymized, structured zero-shot prompt was submitted to the 3 models. The multidisciplinary tumor board decision served as the reference comparator. Overall concordance and concordance for the chemotherapy backbone, maintenance or targeted-agent class, and surgical indication were compared between models using Cochran's Q test and pairwise McNemar tests (Holm-adjusted). Discordant recommendations were categorized as true hallucinations, unsafe, or suboptimal but guideline-concordant.
RESULTS: Concordance varied substantially across models (Cochran's Q p<.001): Gemini 77.3% (95% confidence interval 62.2 to 88.5), Microsoft Copilot 36.4% (95% confidence interval 22.4 to 52.2), and ChatGPT 18.2% (95% confidence interval 8.2 to 32.7); Gemini differed significantly from the other 2 models (Holm-adjusted p<.001), whereas the difference between Microsoft Copilot and ChatGPT did not reach significance after adjustment (p=.057). Component-level analysis localized the divergence to systemic therapy: surgical-indication agreement was uniformly high (93.2% to 95.5%, p=.85). All 3 models generated unsafe recommendations for the re-use of poly (ADP-ribose) polymerase) inhibitors in patients with documented prior progression on poly (ADP-ribose) polymerase inhibitor maintenance, with marked differences in frequency (Microsoft Copilot 10, ChatGPT 3, and Gemini 2; 15 of 74 total discordances). The remaining discordances were suboptimal but guideline concordant.
CONCLUSIONS: Consumer large language models showed heterogeneous and clinically meaningful differences in reproducing Multidisciplinary Tumor Board decisions in recurrent ovarian cancer; none were free of unsafe recommendations. Their current performance does not support independent clinical use, and expert supervision is required to ensure patient safety.
METHODS: A retrospective observational analysis was conducted on 44 consecutive patients with a first recurrence of high-grade serous ovarian cancer treated at a tertiary referral center. For each case, an identical, anonymized, structured zero-shot prompt was submitted to the 3 models. The multidisciplinary tumor board decision served as the reference comparator. Overall concordance and concordance for the chemotherapy backbone, maintenance or targeted-agent class, and surgical indication were compared between models using Cochran's Q test and pairwise McNemar tests (Holm-adjusted). Discordant recommendations were categorized as true hallucinations, unsafe, or suboptimal but guideline-concordant.
RESULTS: Concordance varied substantially across models (Cochran's Q p<.001): Gemini 77.3% (95% confidence interval 62.2 to 88.5), Microsoft Copilot 36.4% (95% confidence interval 22.4 to 52.2), and ChatGPT 18.2% (95% confidence interval 8.2 to 32.7); Gemini differed significantly from the other 2 models (Holm-adjusted p<.001), whereas the difference between Microsoft Copilot and ChatGPT did not reach significance after adjustment (p=.057). Component-level analysis localized the divergence to systemic therapy: surgical-indication agreement was uniformly high (93.2% to 95.5%, p=.85). All 3 models generated unsafe recommendations for the re-use of poly (ADP-ribose) polymerase) inhibitors in patients with documented prior progression on poly (ADP-ribose) polymerase inhibitor maintenance, with marked differences in frequency (Microsoft Copilot 10, ChatGPT 3, and Gemini 2; 15 of 74 total discordances). The remaining discordances were suboptimal but guideline concordant.
CONCLUSIONS: Consumer large language models showed heterogeneous and clinically meaningful differences in reproducing Multidisciplinary Tumor Board decisions in recurrent ovarian cancer; none were free of unsafe recommendations. Their current performance does not support independent clinical use, and expert supervision is required to ensure patient safety.