Algorithm Selection for Estimating Causal Effects: Nulliparous Pregnancy Outcomes Study: Monitoring Mothers to Be.

BACKGROUND

The Super Learner is an ensemble learning method that has been widely used with doubly robust causal effect estimators. It is recommended to deploy the Super Learner with a diverse library of algorithms. To our knowledge, however, the magnitude of the improvements gained by including many algorithms has not yet been systematically evaluated in common epidemiologic research settings.

METHODS

We applied Super Learning with two doubly robust estimators, augmented inverse probability weighting (AIPW) and targeted minimum loss-based estimation (TMLE), to estimate the average treatment effect (ATE) of high periconceptional dietary fruit and vegetable density on the risk of preeclampsia among 7,923 women from the nuMoM2b study. Using a reference ensemble with a diverse library of algorithms, we compared estimates under different sets of algorithms included in the Super Learner to evaluate whether ATE estimates were sensitive to library choices.

RESULTS

The doubly robust estimators fitted with the reference Super Learner ensemble suggested ≥2.5 cups/1,000 kcal of total fruit and vegetable density was associated with a lower risk of preeclampsia. ATE estimated on the risk difference scale by AIPW was -0.019 (95% confidence interval = -0.036, -0.003) and by TMLE was -0.023 (95% confidence interval = -0.039, -0.007). Excluding any individual algorithm from the reference ensemble had little impact on estimates from either AIPW or TMLE. However, relying on a single algorithm (e.g., extreme gradient boosting) yielded results that were much more variable.

CONCLUSION

Our empirical findings support recommendations to build ensemble learners for doubly robust estimators using a diverse array of flexible machine learning algorithms.