Zhejiang da xue xue bao. Yi xue ban = Journal of Zhejiang University. Medical sciences · 2026
[Evaluating the performance of generative AI in assisting the differential diagnosis of weight loss].
Аннотация на английскомСтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Liu Ying, Zhang Yunhong, Cai Dongping, Ren Jingjing
Аннотация
OBJECTIVES: To systematically evaluate the performance of generative artificial intelligence (GenAI) models, DeepSeek-V3 and the Qwen3 series, in the differential diagnosis of weight loss.
METHODS: A search was conducted in the PubMed database for all case reports published in the American Journal of Case Reports between January 1, 2012 and June 2, 2025, containing the term "weight loss" in the title or abstract. Two senior general practitioners independently reviewed each case to determine whether it met predefined diagnostic criteria for weight loss (emaciation). Cases that did not meet these criteria, had incomplete information, or involved clearly defined specialty-specific diagnoses and treatments were excluded. The remaining cases were then compiled into standardized clinical case summaries. These summaries were presented to DeepSeek-V3 and the Qwen3 series models (Qwen3-235B-A22B, Qwen3-30B-A3B, and Qwen3-32B) to generate ranked lists of the top 10 differential diagnoses. The models were not specifically fine-tuned for this task. Sensitivity, precision, and F1-score were used to evaluate performance. Intergroup comparisons were performed using McNemar's test and Cochran's Q test.
RESULTS: A total of 87 case were analyzed. DeepSeek-V3 demonstrated better performance than Qwen3-235B-A22B in sensitivity, precision, and F1-score, especially at the Top5 level (P=0.043). Among the Qwen3 series models, Qwen3-235B-A22B showed the best performance in sensitivity, precision, and F1-score for the Top1 diagnosis, but the differences among the three Qwen3 models across all diagnostic levels were not statistically significant (all P>0.05).
CONCLUSIONS: Domestic GenAI models exhibit a characteristic of "breadth over precision" in the differential diagnosis of weight loss, with DeepSeek-V3 performing better at key diagnostic levels. Although the sensitivity and precision for the top-ranked diagnosis require improvement, these models have the potential to serve as effective clinical decision support tools, broadening the diagnostic perspectives of general practitioners.
METHODS: A search was conducted in the PubMed database for all case reports published in the American Journal of Case Reports between January 1, 2012 and June 2, 2025, containing the term "weight loss" in the title or abstract. Two senior general practitioners independently reviewed each case to determine whether it met predefined diagnostic criteria for weight loss (emaciation). Cases that did not meet these criteria, had incomplete information, or involved clearly defined specialty-specific diagnoses and treatments were excluded. The remaining cases were then compiled into standardized clinical case summaries. These summaries were presented to DeepSeek-V3 and the Qwen3 series models (Qwen3-235B-A22B, Qwen3-30B-A3B, and Qwen3-32B) to generate ranked lists of the top 10 differential diagnoses. The models were not specifically fine-tuned for this task. Sensitivity, precision, and F1-score were used to evaluate performance. Intergroup comparisons were performed using McNemar's test and Cochran's Q test.
RESULTS: A total of 87 case were analyzed. DeepSeek-V3 demonstrated better performance than Qwen3-235B-A22B in sensitivity, precision, and F1-score, especially at the Top5 level (P=0.043). Among the Qwen3 series models, Qwen3-235B-A22B showed the best performance in sensitivity, precision, and F1-score for the Top1 diagnosis, but the differences among the three Qwen3 models across all diagnostic levels were not statistically significant (all P>0.05).
CONCLUSIONS: Domestic GenAI models exhibit a characteristic of "breadth over precision" in the differential diagnosis of weight loss, with DeepSeek-V3 performing better at key diagnostic levels. Although the sensitivity and precision for the top-ranked diagnosis require improvement, these models have the potential to serve as effective clinical decision support tools, broadening the diagnostic perspectives of general practitioners.