Harnessing artificial intelligence for scalable evidence synthesis in reviews: Application in a bibliometric analysis of physical activity technologies.

INTRODUCTION

Artificial intelligence (AI) tools offer promising opportunities to support evidence synthesis at scale. This study presents a novel AI-human hybrid screening approach to a large-scale bibliometric analysis of technologies promoting physical activity.

METHODS

Records (n = 28,957) were retrieved from electronic databases and screened using ASReview, an open-source machine learning tool. Over 100 seed articles trained the model. Screening followed the SAFE framework across four phases, including (1) initial random screening to inform stopping rules, (2) active learning with human reviewers, and multi-model rescreening of (3) unlabelled and (4) excluded records to minimise risk of missed studies.

RESULTS

In Phase 1, a random 1% sample (n = 290) was screened, identifying 20 relevant records. In Phase 2, 3,994 records were screened using active screening, identifying 2,904 relevant studies. In Phase 3, re-screening of unlabelled records (n = 410) identified 53 additional studies, while Phase 4 re-evaluation of excluded records yielded a further 226 studies. Across all phases, 3,183 records were identified as relevant, with 2,985 retained for analysis following post-screening exclusions (n = 598). Only 18% of records required manual screening, saving an estimated 592 hours.

CONCLUSION

AI-assisted screening offers a feasible and efficient approach for large-scale evidence synthesis when supported by structured workflows and safeguards. While methods like careful seed selection and stopping rules improve rigour, challenges remain-particularly residual risks and reliance on manual data extraction. Future work should focus on extending AI to downstream tasks and embedding human-in-the-loop approaches to ensure it serves as a reliable, transparent partner in evidence synthesis.