Arthroplasty today · 2026
Evaluating the Utility of Artificial Intelligence in Conducting Systematic Reviews.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Carrier Robert, Lopez Leonardo, Moya Andrew, Hernandez Victor H, D'Apuzzo Michele R, McNamara Colin A
Аннотация
BACKGROUND: Systematic reviews and meta-analyses represent the highest level of evidence in clinical research, but the process of article retrieval and screening is labor-intensive. Large language models, such as ChatGPT-5, may offer an efficient alternative, yet their performance in full systematic review workflows remains untested. This study compares ChatGPT-5's Deep Research and Agent Modes with human researchers in replicating gold standard systematic reviews in total joint arthroplasty.
METHODS: Five published systematic reviews were selected as reference articles. Three groups: orthopaedic research fellows, ChatGPT-5 Deep Research Mode, and ChatGPT-5 Agent Mode, independently identified eligible articles using standardized search terms and inclusion/exclusion criteria. Artificial intelligence (AI) searches were repeated 3 times for reproducibility. Extracted articles were evaluated against the gold standard for recall, precision, false positives/negatives, and time efficiency. Newly identified eligible studies were also assessed.
RESULTS: The research fellows dedicated 268 hours to screening 9101 articles, achieving 85.2% recall of gold standard articles. Deep Research and Agent Modes averaged 12-14 minutes per search, identifying 47.5% and 40.9% of gold standard articles, respectively. Fellows had fewer false negatives (n = 5) compared with Deep Research (n = 19) and Agent Mode (n = 12). AI models retrieved several additional eligible studies not captured by humans, demonstrating complementary potential.
CONCLUSIONS: Human reviewers remain superior to current AI models in replicating systematic review article selection, particularly for nuanced inclusion/exclusion criteria. However, ChatGPT-5 significantly reduces search time and can identify additional relevant studies, suggesting its role as a valuable adjunct in systematic review workflows with expert oversight.
METHODS: Five published systematic reviews were selected as reference articles. Three groups: orthopaedic research fellows, ChatGPT-5 Deep Research Mode, and ChatGPT-5 Agent Mode, independently identified eligible articles using standardized search terms and inclusion/exclusion criteria. Artificial intelligence (AI) searches were repeated 3 times for reproducibility. Extracted articles were evaluated against the gold standard for recall, precision, false positives/negatives, and time efficiency. Newly identified eligible studies were also assessed.
RESULTS: The research fellows dedicated 268 hours to screening 9101 articles, achieving 85.2% recall of gold standard articles. Deep Research and Agent Modes averaged 12-14 minutes per search, identifying 47.5% and 40.9% of gold standard articles, respectively. Fellows had fewer false negatives (n = 5) compared with Deep Research (n = 19) and Agent Mode (n = 12). AI models retrieved several additional eligible studies not captured by humans, demonstrating complementary potential.
CONCLUSIONS: Human reviewers remain superior to current AI models in replicating systematic review article selection, particularly for nuanced inclusion/exclusion criteria. However, ChatGPT-5 significantly reduces search time and can identify additional relevant studies, suggesting its role as a valuable adjunct in systematic review workflows with expert oversight.