Limitations in conduct and reporting of cochrane reviews rarely inhibit the determination of the validity of evidence for clinical decision-making.

OBJECTIVE

To determine how often clinical conclusions derived from Cochrane Reviews have uncertain validity due to review conduct and reporting deficiencies.

METHODS

We evaluated 5142 clinical conclusions in DynaMed (an evidence-based point-of-care clinical reference) based on 4743 Cochrane Reviews. Clinical conclusions with level 2 evidence due to shortcomings in the review's conduct or reporting (rather than deficiencies in the underlying evidence) were confirmed by a DynaMed editor and two Cochrane Review authors.

RESULTS

Thirty-one Cochrane Reviews (0.65%) had confirmed deficiencies in conduct and reporting as the reason for classifying 37 assessed clinical conclusions (0.72%) as level 2 evidence. In all cases, it was not feasible for the assessors to specify a clear criticism of the studies included in the reviews. The deficiencies were specific to not accounting for dropouts (2) or inadequate assessment and reporting of allocation concealment (11), other specific trial quality criteria (14), or all trial quality criteria (4).

CONCLUSIONS

Cochrane Reviews provide high-quality assessment and synthesis of evidence, with fewer than 1% of Cochrane Reviews having limitations which hinder the summary of best current evidence for clinical decision-making. We expect this will further decrease following recent Cochrane quality initiatives.