Computers in biology and medicine · 2026
External validation of GDM risk prediction models using a machine learning reciprocal model-exchange framework.
СтатьяВалидационное исследованиеНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Germaine Mark, Belsti Yitayeh, O'Higgins Amy, Egan Brendan, Teede Helena, Healy Graham, Enticott Joanne
Аннотация
BACKGROUND: Although many risk prediction models have been developed, very few undergo external validation, primarily due to issues with data access. Therefore, we implemented a reciprocal model-exchange approach to facilitate external validation and demonstrate its use with gestational diabetes mellitus (GDM) prediction models.
OBJECTIVE: To assess the robustness and generalisability of two independently developed GDM risk prediction models using a reciprocal model-exchange framework.
METHODS: Two independently developed GDM risk prediction models were externally validated using a reciprocal model-exchange. The saved model's corresponding variable types and data pre-processor were exchanged. The Monash CatBoost model was validated using Irish data at Dublin City University (DCU), and the DCU logistic-regression GDM model was validated using Australian data at Monash University. Performance was assessed using discrimination, calibration and decision curve analysis. Model fairness was assessed.
RESULTS: The prevalence of GDM was 21.1% in the Australian cohort and 11.7% in the Irish cohort. The Monash model's AUC dropped from 0.93 to 0.77, while the DCU model's AUC fell from 0.82 to 0.69. Calibration estimates confirmed systematic risk misestimation; each model tends to over or under-predict GDM probabilities outside its training domain, with calibration-in-the-large of -0.573 for the Monash model and 0.17 for the DCU model; slopes were 1.278 and 0.55 respectively. Both models showed performance variability across ethnic groups, with lower performance for Southeast/Northeast Asians and both performed better with increasing parity and among women without a prior GDM diagnosis.
CONCLUSIONS: Each model's performance decreased upon external validation, and the fairness evaluations on the different sub-categories (ethnicities; parity and previous GDM) provided evidence on the areas to be addressed in model recalibration/updating before deployment can be progressed. This reciprocal model-exchange approach provides a solution to facilitating external validations, which are notably lacking in the current literature but are necessary to advance the risk prediction field.
OBJECTIVE: To assess the robustness and generalisability of two independently developed GDM risk prediction models using a reciprocal model-exchange framework.
METHODS: Two independently developed GDM risk prediction models were externally validated using a reciprocal model-exchange. The saved model's corresponding variable types and data pre-processor were exchanged. The Monash CatBoost model was validated using Irish data at Dublin City University (DCU), and the DCU logistic-regression GDM model was validated using Australian data at Monash University. Performance was assessed using discrimination, calibration and decision curve analysis. Model fairness was assessed.
RESULTS: The prevalence of GDM was 21.1% in the Australian cohort and 11.7% in the Irish cohort. The Monash model's AUC dropped from 0.93 to 0.77, while the DCU model's AUC fell from 0.82 to 0.69. Calibration estimates confirmed systematic risk misestimation; each model tends to over or under-predict GDM probabilities outside its training domain, with calibration-in-the-large of -0.573 for the Monash model and 0.17 for the DCU model; slopes were 1.278 and 0.55 respectively. Both models showed performance variability across ethnic groups, with lower performance for Southeast/Northeast Asians and both performed better with increasing parity and among women without a prior GDM diagnosis.
CONCLUSIONS: Each model's performance decreased upon external validation, and the fairness evaluations on the different sub-categories (ethnicities; parity and previous GDM) provided evidence on the areas to be addressed in model recalibration/updating before deployment can be progressed. This reciprocal model-exchange approach provides a solution to facilitating external validations, which are notably lacking in the current literature but are necessary to advance the risk prediction field.