Flipping death-to-survival event coding shifted pooled risk ratios and GRADE imprecision ratings in 2,071 Cochrane meta-analyses.

OBJECTIVE: When meta-analyzing a dichotomous outcome such as death or survival, reviewers must decide which side of the outcome to code as the event. In a single study, death and survival are two sides of the same dichotomous outcome, and the choice does not change the result. However, in a meta-analysis using the risk ratio, this choice affects the weight given to each study. Pooling deaths and pooling survivors need not give equivalent results. We evaluated how flipping the event coding of death or survival outcomes affects pooled risk ratios, precision, and GRADE imprecision judgments across Cochrane reviews.

STUDY DESIGN AND SETTING: We conducted a meta-epidemiological study of Cochrane reviews with death or survival outcomes that used risk ratios or odds ratios as the effect measure. For each meta-analysis, we created a flipped version of the reported 2 x 2 data by replacing event counts with total minus events. We then calculated pooled risk ratios for the original and flipped data using random-effects Mantel-Haenszel methods. The flipped pooled risk ratio was re-expressed as its reciprocal on the original scale and compared with the original as a ratio of risk ratios. We compared statistical significance, confidence intervals, and GRADE imprecision ratings between original and flipped analyses. We repeated the analysis using odds ratios and examined subgroups by total sample size, baseline control event rate, and single-zero or double-zero studies.

RESULTS: A total of 2,071 meta-analyses were included. The median ratio of risk ratios was 1.17 (IQR 1.05 to 1.42), showing that flipping moved the pooled risk ratio toward the null in most meta-analyses. Statistical significance changed in 199 (9.6%). GRADE imprecision changed in 1,211 (58%): worsening in 672 (32%) and improving in 539 (26%). Using odds ratios instead of risk ratios showed a far smaller divergence. The discrepancies were apparent in meta-analyses with single-zero or double-zero studies, low baseline control event rates, small total sample sizes, or large original effect sizes.

CONCLUSION: Flipping the event coding of death or survival outcomes changed risk ratio results and GRADE imprecision judgments considerably, especially in sparse or small meta-analyses. Systematic reviewers should use odds ratios instead of risk ratios when primary studies did not originally use a consistent event coding.

PLAIN LANGUAGE SUMMARY: When researchers combine results from multiple clinical trials (a meta-analysis) to see whether a treatment changes the risk of death, they must first decide whether to count deaths or survivors as the "event." In a single trial, this choice does not matter. We found that in a meta-analysis, it can matter a great deal: recoding deaths as survivors (or vice versa) changed the combined result, whether it was statistically significant, and how confident reviewers should be in the finding, in a large proportion of the 2,071 meta-analyses we examined. This was especially true when some trials had very few or no deaths in one group. Using the odds ratio instead of the risk ratio avoided this problem almost completely. Systematic reviewers should keep this in mind when trials report death and survival inconsistently.