BMJ evidence-based medicine · 2025
Post-publication peer review and the identification of methodological and reporting issues in COVID-19 trials: a qualitative study.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Davidson Mauricia, Korfitsen Christoffer Bruun, Riveros Carolina, Chaimani Anna, Boutron Isabelle
Аннотация
OBJECTIVES: We aimed to determine to what extent systematic reviewers and post-preprint and post-publication peer review identified methodological and reporting issues in COVID-19 trials that could be easily resolved by the authors.
DESIGN: Qualitative study.
DATA SOURCES: COVID-NMA living systematic review (covid-nma.com), PubPeer, medRxiv, Research Square, SSRN.
METHODS: We considered randomised controlled trials (RCTs) in COVID-NMA that evaluated pharmacological treatments for COVID-19 and retrieved systematic reviewers' assessments of the risk of bias and outcome reporting bias. We also searched for commentary data on PubPeer and preprint servers up to 6 November 2023. We employed qualitative content analysis to develop themes and domains of methodological and reporting issues identified by commenters.
RESULTS: We identified 500 eligible RCTs. Systematic reviewers identified methodological and reporting issues in 446 (89%) RCTs. In 391 (78%) RCTs, the issues could be easily resolved by the trial authors; issues included incomplete reporting (49%), selection of the reported results (52%) and no access to the pre-specified plan (25%). Alternatively, 74 (15%) RCTs had received at least one comment on PubPeer or preprint servers, totalling 348 comments. In 46 (9%) RCTs, the issues identified by post-preprint and post-publication peer review comments could be easily resolved by the trial authors; the issues were related to incomplete reporting (6%), errors (5%), statistical analysis (3%), inconsistent reporting of methods and analyses (2%), spin (2%), selection of the reported results (1%) and no access to the raw data/pre-specified plan (1%).
CONCLUSIONS: Without changing their process, systematic reviewers identified issues in most RCTs that could be easily resolved by the trial authors; however, the lack of an established author feedback mechanism represents a wasted opportunity for facilitating improvement and enhancing the overall manuscript quality. On the other hand, despite the existing feedback loop to authors present in post-publication peer review, it demonstrated limited effectiveness in identifying methodological and reporting issues.
DESIGN: Qualitative study.
DATA SOURCES: COVID-NMA living systematic review (covid-nma.com), PubPeer, medRxiv, Research Square, SSRN.
METHODS: We considered randomised controlled trials (RCTs) in COVID-NMA that evaluated pharmacological treatments for COVID-19 and retrieved systematic reviewers' assessments of the risk of bias and outcome reporting bias. We also searched for commentary data on PubPeer and preprint servers up to 6 November 2023. We employed qualitative content analysis to develop themes and domains of methodological and reporting issues identified by commenters.
RESULTS: We identified 500 eligible RCTs. Systematic reviewers identified methodological and reporting issues in 446 (89%) RCTs. In 391 (78%) RCTs, the issues could be easily resolved by the trial authors; issues included incomplete reporting (49%), selection of the reported results (52%) and no access to the pre-specified plan (25%). Alternatively, 74 (15%) RCTs had received at least one comment on PubPeer or preprint servers, totalling 348 comments. In 46 (9%) RCTs, the issues identified by post-preprint and post-publication peer review comments could be easily resolved by the trial authors; the issues were related to incomplete reporting (6%), errors (5%), statistical analysis (3%), inconsistent reporting of methods and analyses (2%), spin (2%), selection of the reported results (1%) and no access to the raw data/pre-specified plan (1%).
CONCLUSIONS: Without changing their process, systematic reviewers identified issues in most RCTs that could be easily resolved by the trial authors; however, the lack of an established author feedback mechanism represents a wasted opportunity for facilitating improvement and enhancing the overall manuscript quality. On the other hand, despite the existing feedback loop to authors present in post-publication peer review, it demonstrated limited effectiveness in identifying methodological and reporting issues.