Journal of clinical epidemiology · 2026
Evaluating the accuracy of full text and serial screening by one versus two independent reviewers: Part 2 of a meta-research study of rapid review methodological trade-offs.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Corso Melissa, Ead Lauren, Brunton Ginny, Negrini Stefano, Tricco Andrea C, Côté Pierre
Аннотация
OBJECTIVE: We aimed to determine the accuracy of one versus two reviewers for full-text screening, and serial screening (title/abstract and full text). We also determined the inter-rater reliability, post-test probability (PTP), false positive and negative rates, and sources of misclassification.
DESIGN AND SETTING: One reviewer screened fulltext articles from three systematic reviews evaluating interventions (acupuncture, education and TENS) for chronic low back pain. We combined these results with titles and abstracts screened by the same reviewer to assess serial screening accuracy (screening title and abstracts then full text studies). Outcomes were compared to consensus decisions from two independent reviewers. We computed sensitivity, specificity, positive (PPV) and negative predictive values (NPV) with 95% confidence intervals (CIs). Inter-rater reliability (Cohen's kappa) and positive PTP for serial screening were calculated. Rates and reasons for misclassification were examined. A sensitivity analysis of English studies explored the impact of AI-assisted translation of non-English studies.
RESULTS: A total of 120, 92 and 91 studies were screened for the acupuncture, education, and TENS reviews, respectively. Compared with two reviewers, the sensitivity of single reviewer full-text screening ranged from 62% to 88%, specificity from 84% to 96%, PPV from 70% to 77% and NPV from 89% to 97%. Kappa ranged from 0.61 to 0.73. Single reviewer sensitivity for serial screening ranged from 62% to 81%, specificity from 98% to 100%, PPV from 68% to 71% and NPV was 99%. Kappa ranged from 0.67 to 0.73. Positive PTP ranged from 42.7% to 63.5%. Misclassifications were mostly due to interpretation of the population, interventions or comparators. There were no or small differences in the estimates with overlap of CIs between English and all language analyses.
CONCLUSIONS: The accuracy of one versus two reviewers may be reduced in serial screening, resulting in missed and falsely included studies, ranging across reviews. Therefore, there is uncertainty around whether a single reviewer can correctly select relevant studies. However, the impact on rapid review results is unknown. Although overall agreement is substantial, reporting sensitivity and specificity can provide insight into classifications between reviewers. Carefully defining the population, intervention and comparators in screening is required and highlights the need for improved reporting in trials.
DESIGN AND SETTING: One reviewer screened fulltext articles from three systematic reviews evaluating interventions (acupuncture, education and TENS) for chronic low back pain. We combined these results with titles and abstracts screened by the same reviewer to assess serial screening accuracy (screening title and abstracts then full text studies). Outcomes were compared to consensus decisions from two independent reviewers. We computed sensitivity, specificity, positive (PPV) and negative predictive values (NPV) with 95% confidence intervals (CIs). Inter-rater reliability (Cohen's kappa) and positive PTP for serial screening were calculated. Rates and reasons for misclassification were examined. A sensitivity analysis of English studies explored the impact of AI-assisted translation of non-English studies.
RESULTS: A total of 120, 92 and 91 studies were screened for the acupuncture, education, and TENS reviews, respectively. Compared with two reviewers, the sensitivity of single reviewer full-text screening ranged from 62% to 88%, specificity from 84% to 96%, PPV from 70% to 77% and NPV from 89% to 97%. Kappa ranged from 0.61 to 0.73. Single reviewer sensitivity for serial screening ranged from 62% to 81%, specificity from 98% to 100%, PPV from 68% to 71% and NPV was 99%. Kappa ranged from 0.67 to 0.73. Positive PTP ranged from 42.7% to 63.5%. Misclassifications were mostly due to interpretation of the population, interventions or comparators. There were no or small differences in the estimates with overlap of CIs between English and all language analyses.
CONCLUSIONS: The accuracy of one versus two reviewers may be reduced in serial screening, resulting in missed and falsely included studies, ranging across reviews. Therefore, there is uncertainty around whether a single reviewer can correctly select relevant studies. However, the impact on rapid review results is unknown. Although overall agreement is substantial, reporting sensitivity and specificity can provide insight into classifications between reviewers. Carefully defining the population, intervention and comparators in screening is required and highlights the need for improved reporting in trials.