[A comparative study on the application of DeepSeek-R1 and ChatGPT in multidisciplinary treatment decision-making for advanced gastric cancer].

Objective

To compare the accuracy and comprehensiveness of DeepSeek-R1 and ChatGPT-4o in generating treatment recommendations for advanced gastric cancer.

Methods

This study included three steps: (1) evaluating the answers to ten key clinical questions; (2) analyzing clinical cases from the multidisciplinary team (MDT) of our center; (3) reviewing rare gastric cancer cases on PubMed. The study cases included MDT data of 95 patients with advanced gastric cancer treated at the Second Affiliated Hospital of Nanchang University from November 2022 to July 2024, as well as 14 rare cases retrieved from PubMed. Prompts designed based on the advanced gastric cancer cases were submitted to DeepSeek-R1 and ChatGPT-4o in a standardized format. A structured 4-point Likert scale was used to evaluate the accuracy and completeness of the outputs. Inter-rater consistency was calculated to ensure the objectivity of the evaluation.

Results

DeepSeek-R1 outperformed ChatGPT-4o in both accuracy and completeness regarding the ten key clinical questions, the practical MDT cases from our center, and the rare cases from PubMed. Stratified analysis showed that DeepSeek-R1 had advantages in providing answers related to surgical recommendations, chemotherapy suggestions, and chemotherapy regimens. The evaluation of inter-rater reliability revealed high reliability among raters (Accuracy and completeness: For key clinical questions: W=0.696 and W=0.632, respectively; For practical MDT cases of our center: W=0.657 and W=0.634, respectively; For rare cases from PubMed: W=0.683 for accuracy; all P<0.001).

Conclusion

DeepSeek-R1 demonstrates slightly better performance than ChatGPT-4o in generating treatment recommendations for advanced gastric cancer cases.