Assessing the impact of unpublished data on network meta-analysis outcomes in outpatient adults with acute migraine: a study within a review.

BACKGROUND AND

OBJECTIVES

Methodological guidance recommends including both published and unpublished data in systematic reviews to enhance reliability and reduce potential bias. The objective of the study was to evaluate the impact of unpublished data from double-blind, pharmacologic randomized controlled trials on the results of network meta-analysis (NMA) of migraine treatments.

METHODS

We supplemented the search of a recent systematic review with targeted searches for unpublished data on ClinicalTrials.gov, the World Health Organization International Clinical Trials Registry Platform, regulatory agency reports, The Preprints Citation Index, Europe PubMed Central, and Embase.com, and conference abstracts from the past 5 years. Two independent reviewers selected eligible studies, verified publication status, extracted data, and reassessed certainty of evidence (COE). We reproduced the original NMA including unpublished data for four dichotomous outcomes. We compared our results with the original NMA in terms of risk ratio (RR), absolute risk difference with 95% CI, COE assessments, and conclusions.

RESULTS

Seventeen (6735 participants) of 37 eligible unpublished trials had posted results and were analyzed, with most (59%) identified via trial registries. In addition, unpublished outcome data were retrieved for four published trials from the original analysis. These unpublished trials added two previously unrepresented interventions to the NMA, increasing the number of direct comparisons and closed loops. Comparisons of RRs (95% CI) with the original analysis showed all effects maintained the same direction, although six CIs newly crossed the null effect (RR = 1). Among 144 COE assessments, four changed meaningfully: one was rated down from high to moderate due to imprecision, and three were rated up from very low/no evidence to low COE based on new direct evidence. Overall conclusions remained unchanged after including unpublished data.

CONCLUSIONS

In this case study, adding unpublished data had minimal impact on results and conclusions, with only minor changes in network geometry and COE.