International journal of medical informatics · 2025
"I don't know": An uncertainty-aware machine learning model for predicting patient disposition at emergency department triage.
СтатьяПоддержка исследования: вне правительства СШАНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Abdulai Abubakar Sadiq Bouda, Storm Jean, Ehrlich Michael
Аннотация
BACKGROUND: Machine learning (ML) models are widely used for predicting patient disposition at emergency department (ED) triage. However, these models generate predictions regardless of the level of uncertainty, potentially leading to overconfident outputs that can compromise clinical decision-making.
OBJECTIVE: To develop a conformal prediction model for ED triage that provides uncertainty-aware patient disposition predictions.
METHODS: This retrospective study analyzed 560,486 adult ED visits (March 2014 - July 2017) from one academic and two community hospitals. An extreme gradient boosting (XGBoost) model was trained, validated, and conformalized to introduce a "Don't know" prediction for high-uncertainty cases. The model was tested on a random sample of 56,000 ED cases.
RESULTS: The standard XGBoost model achieved an AUC of 0.9307 (95% CI: 0.9285 - 0.9329), with sensitivity of 0.72 and specificity of 0.94. With conformal prediction at a lower confidence threshold of 60%, the model indicated "Don't know" in 4.9% of cases while returning sensitivity and specificity values of 0.74 and 0.95, respectively. As confidence thresholds increased, the model returned more "Don't know" predictions and fewer misclassifications. At 90% confidence, the model returned "Don't know" in 34.5% of cases while returning sensitivity and specificity values of 0.88 and 0.99, respectively. This trade-off highlights a balance between model confidence and prediction accuracy.
CONCLUSION: Incorporating uncertainty-awareness in ML models improves reliability in ED triage. By acknowledging uncertainty, clinicians receive more interpretable insights, reducing the risk of overconfident predictions and enhancing patient safety.
OBJECTIVE: To develop a conformal prediction model for ED triage that provides uncertainty-aware patient disposition predictions.
METHODS: This retrospective study analyzed 560,486 adult ED visits (March 2014 - July 2017) from one academic and two community hospitals. An extreme gradient boosting (XGBoost) model was trained, validated, and conformalized to introduce a "Don't know" prediction for high-uncertainty cases. The model was tested on a random sample of 56,000 ED cases.
RESULTS: The standard XGBoost model achieved an AUC of 0.9307 (95% CI: 0.9285 - 0.9329), with sensitivity of 0.72 and specificity of 0.94. With conformal prediction at a lower confidence threshold of 60%, the model indicated "Don't know" in 4.9% of cases while returning sensitivity and specificity values of 0.74 and 0.95, respectively. As confidence thresholds increased, the model returned more "Don't know" predictions and fewer misclassifications. At 90% confidence, the model returned "Don't know" in 34.5% of cases while returning sensitivity and specificity values of 0.88 and 0.99, respectively. This trade-off highlights a balance between model confidence and prediction accuracy.
CONCLUSION: Incorporating uncertainty-awareness in ML models improves reliability in ED triage. By acknowledging uncertainty, clinicians receive more interpretable insights, reducing the risk of overconfident predictions and enhancing patient safety.