Real-World Implementation and Evaluation of AI-Driven Clinical Decision Support in Emergency Medicine: A Systematic Review.

Background/

Objectives

Emergency departments (EDs) are high-pressure environments where time-sensitive decisions, fragmented data, and operational strain create strong demand for AI-driven clinical decision-support systems (AI-CDSSs). These systems have shown promise in triage, diagnosis, risk stratification, and workflow optimization, yet real-world implementation in emergency medicine remains uneven. This systematic review aimed to synthesize the technical characteristics, clinical applications, implementation dimensions, organizational and ethical considerations, and real-world impact of AI-CDSSs in ED settings.

Methods

This systematic review followed PRISMA guidance and searched PubMed, Scopus, and Embase for English-language studies published between January 2015 and February 2025. Eligible studies described AI-CDSS implementation, clinical integration, or performance evaluation in ED settings. Twenty-three studies met the inclusion criteria and were synthesized across five domains: technical characteristics, clinical applications, implementation dimensions, organizational and ethical considerations, and real-world impact.

Results

Among the 23 included studies, 20 contributed to the real-world evaluation synthesis. Of these 20 studies, 8 (40%) achieved live or prospective evaluation, seven (35%) relied only on retrospective validation, four (20%) used human-centered or perception-based evaluation, two (10%) used post-implementation assessment, and one (5%) used simulation-based evaluation; categories were not mutually exclusive because some studies employed more than one evaluation approach. In the separate clinical-application synthesis of 20 studies, AI-CDSS were most frequently applied to diagnosis and immediate intervention (45%), followed by prediction and risk stratification (35%) and operational improvement (20%). Successful adoption was more consistently associated with EHR integration, workflow-sensitive design, and clinician engagement than with algorithmic performance alone. Persistent barriers included limited external validation, weak drift-monitoring plans, inconsistent usability testing, regulatory ambiguity, and insufficient equity mitigation.

Conclusions

Sustainable implementation of AI-CDSSs in emergency medicine will require prospective multi-site evaluation, sociotechnical integration, adaptive governance, and greater attention to equity. Technical performance alone is insufficient to establish clinical readiness; successful implementation also depends on integration with clinical workflows, clinician engagement, ongoing monitoring, and appropriate governance.