The Fragility of Statistical Findings in Distal Radius Fractures: A Systematic Review of Randomized Controlled Trials.

PURPOSE: Probability analysis with reporting of P values is often used to determine the statistical significance of study findings in the orthopedic literature. The fragility index (FI) has been used to evaluate the robustness of a significant result. The purpose of this study was to determine the utility of applying a fragility analysis to randomized controlled trials (RCTs) evaluating distal radius fractures (DRFs).

METHODS

We identified all dichotomous outcome data for randomized controlled trials of distal radius fractures (DRFs) published in 13 orthopedic journals from 2000 to 2020. The fragility index (FI) was determined by the number of event reversals required to change a P value from less than 0.05 to greater than 0.05, or vice-versa. The associated fragility quotient (FQ) was determined by dividing the FI by the sample size. The interquartile ranges (IQR) for the FI and FQ were calculated as the difference between the 25th and 75th percentiles for the overall study.

RESULTS

Of the 7352 studies screened, 221 met the search criteria, with 34 randomized controlled trials evaluating distal radius fractures and 151 total outcome events included for analysis. The overall FI was 9 (IQR 6.5-11) with an associated fragility quotient of 0.097 (IQR 0.060-0.140). However, a majority of outcomes (78.8%) were not significant. Fragility analysis of statistically significant outcomes had an FI of 4 (IQR 1-10). All of the studies reported lost to follow-up (LTF) data where 20.6% reported lost to follow-up of greater or equal to 9.

CONCLUSION

The RCTs in distal radius fracture outcomes have comparable statistical robustness to literature in other orthopedic surgical specialties, are not as statistically stable as previously thought and should be interpreted with caution. We recommend that orthopedic literature report the FI and FQ along with the P values to help the reader draw reliable conclusions based on the fragility of outcome measures.

LEVEL OF EVIDENCE: Level I.