The importance of accounting for the uncertainty of published prognostic model estimates.

OBJECTIVES

Reported is the importance of properly reflecting uncertainty associated with prognostic model estimates when calculating the survival benefit of a treatment or technology, using liver transplantation as an example.

METHODS

Monte Carlo simulation techniques were used to account for the uncertainty of prognostic model estimates using the standard errors of the regression coefficients and their correlations. These methods were applied to patients with primary biliary cirrhosis undergoing liver transplantation using a prognostic model from a historic cohort who did not undergo transplantation. The survival gain over 4 years from transplantation was estimated.

RESULTS

Ignoring the uncertainty in the prognostic model, the estimated survival benefit of liver transplantation was 16.7 months (95 percent confidence interval [CI], 13.5 to 20.1), and was statistically significant (p < .001). After adjusting for model uncertainty using the standard errors of the regression coefficients, the estimated survival benefit was 17.5 months (95 percent CI, -3.9 to 38.5) and was no longer statistically significant. An additional adjustment for the correlation between regression coefficients widened the 95 percent confidence interval slightly: the estimated survival benefit was 17.0 months (95 percent CI: -4.6 to 38.6).

CONCLUSIONS

It is important that the precision of regression coefficients is available for users of published prognostic models. Ignoring this additional information substantially underestimates uncertainty, which can then impact misleadingly on policy decisions.