Evaluating the Performance of Traditional Pharmacoepidemiologic and Machine Learning Models to Predict Pregnancies at Risk of Major Congenital Malformations.

BACKGROUND

With approximately 50% of pregnancies being unplanned, there is an unintended exposure to potential feto-toxic drugs that may cause major congenital malformations (MCM). This study aims to compare the predictive performance between traditional pharmacoepidemiologic (PE) and machine learning (ML) models.

METHODS

We conducted a cohort study within the Quebec Pregnancy Cohort, including all pregnancies covered by Quebec's prescription drug insurance program and their children from 01/1998 to 12/2015. Medication exposures, comorbidities, and women's characteristics 12 months before pregnancy and during the first trimester were considered. Robust Poisson models were used to obtain adjusted risk ratios (aRR) and 95% confidence intervals (CI) of predictors. Logistic regression, robust Poisson, K-Nearest Neighbors, Random Forest, XGBoost, Naïve bayes, Multilayer Perceptron, and Support Vector Machine were developed to predict pregnancies at risk of MCM. Sensitivity, specificity, PPV, NPV, accuracy, ROC-AUCs, PR-AUCs, and F1-score were used to evaluate the performance of predictive models.

RESULTS

We analyzed 213,744 pregnancies, finding a 9.7% prevalence of MCM. Logistic regression had the highest discriminative power across models at predicting MCM, with a ROC-AUC of 53.3% and the highest sensitivity (41.5%) and F1-score (46.6%). KNN had the highest specificity (97.9%) but the lowest sensitivity (2.3%). Robust Poisson performed similarly to logistic regression, with the highest accuracy (52.3%). Robust Poisson performed slightly better than logistic regression at classifying organ-specific malformations. All models showed poor overall predictive performance. Results were robust across sensitivity analyses.

CONCLUSION

There is insufficient evidence for the superiority of ML over traditional pharmacoepidemiologic modeling in predicting MCM.