Journal of clinical epidemiology · 2025
Grading of Recommendations Assessment, Development, and Evaluation and Confidence in Network Meta-Analysis showed moderate to substantial concordance in the evaluation of certainty of the evidence.
СтатьяПоддержка исследования: вне правительства СШАНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Minozzi Silvia, Cinquini Michela, Arienti Chiara, Battain Pier Carlo, Brigadoi Giulia, Del Vicario Miryam, Di Domenico Giulia, Farma Tommaso, Federico Sara, Innocenti Tiziano, La Rosa Giusy Rita Maria, Orlandi Elena, Piersanti Alessandra, Selvanetti Alberto, Zanetta Laura, Del Giovane Cinzia
Аннотация
OBJECTIVES: Grading of Recommendations Assessment, Development, and Evaluation (GRADE) and Confidence in Network Meta-Analysis (CINeMA) are available to assess the confidence in network meta-analysis (NMA) results. They share common aspects, but their operationalization differs. We evaluated interrater reliability (IRR) among assessors, the approaches' concordance, and application time.
METHODS: Two dichotomous ("seizure response", "ischemic stroke") and two continuous ("change in attention deficit hyperactivity disorder (ADHD)", "weight loss") outcomes with networks of different sizes, structures and complexities were chosen from four NMAs. Thirteen assessors were randomly assigned to four groups to apply both GRADE and CINeMA on one continuous and one dichotomous outcome. We measured IRR and concordance using Gwet's AC on the overall evaluation and each tool's domain. We calculated time spent evaluating each network and tool, including time to consult the guidance papers.
RESULTS: IRR ranged from 0.49 ("seizure response") to 0.70 ("ischemic stroke") with GRADE, from 0.02 ("ischemic stroke") to 0.73 ("change in ADHD symptoms") with CINeMA. Overall concordance was 1, 0.90, 0.68, and 0.42 for "seizure response", "ADHD symptoms", "weight loss", and "ischemic stroke", respectively. The median time spent to assess each network ranged from 160 to 481 minutes with GRADE, from 150 to 330 minutes with CINeMA.
CONCLUSION: IRR was moderate to substantial with both approaches, except for CINeMA when applied to the largest and most complex network ("ischemic stroke"). Concordance was good for small networks with no or few indirect comparisons ("seizure response") and decreased as the number of comparisons and indirect evidence increased. Application time was long with both approaches, particularly with GRADE for large networks.
METHODS: Two dichotomous ("seizure response", "ischemic stroke") and two continuous ("change in attention deficit hyperactivity disorder (ADHD)", "weight loss") outcomes with networks of different sizes, structures and complexities were chosen from four NMAs. Thirteen assessors were randomly assigned to four groups to apply both GRADE and CINeMA on one continuous and one dichotomous outcome. We measured IRR and concordance using Gwet's AC on the overall evaluation and each tool's domain. We calculated time spent evaluating each network and tool, including time to consult the guidance papers.
RESULTS: IRR ranged from 0.49 ("seizure response") to 0.70 ("ischemic stroke") with GRADE, from 0.02 ("ischemic stroke") to 0.73 ("change in ADHD symptoms") with CINeMA. Overall concordance was 1, 0.90, 0.68, and 0.42 for "seizure response", "ADHD symptoms", "weight loss", and "ischemic stroke", respectively. The median time spent to assess each network ranged from 160 to 481 minutes with GRADE, from 150 to 330 minutes with CINeMA.
CONCLUSION: IRR was moderate to substantial with both approaches, except for CINeMA when applied to the largest and most complex network ("ischemic stroke"). Concordance was good for small networks with no or few indirect comparisons ("seizure response") and decreased as the number of comparisons and indirect evidence increased. Application time was long with both approaches, particularly with GRADE for large networks.