Agreement and Classification Performance of a Mobile Self-Triage Application Based on the Manchester Triage System in Hemodynamically Stable Adult Walk-In Emergency Department Patients: A Prospective Observational Study.

Background/

Objectives

Digital self-triage tools may support patient-flow management in crowded emergency departments (EDs), but their agreement with established professional triage systems requires prospective validation. We evaluated the agreement and classification performance of a locally developed mobile self-triage application, structured according to Manchester Triage System (MTS)-related urgency logic, compared with professional MTS assessment in adult ED patients.

Methods

This prospective, observational, single-center agreement study included 2274 consecutive, hemodynamically stable adult walk-in patients who used a tablet-based application covering 11 predefined presenting complaints and subsequently underwent a single, blinded professional MTS assessment, which served as the prespecified comparator. Of 3244 initial registrations, 970 (29.9%) were excluded because their complaints were not covered by the predefined pathways. MTS Categories 2-3 were classified as urgent.

Results

Exact agreement occurred in 968 patients (42.6%); 1161 patients (51.1%) were overtriaged and 145 (6.3%) were undertriaged. The application showed 93.9% sensitivity, 28.7% specificity, 61.1% positive predictive value, and 79.8% negative predictive value for identifying professional MTS-defined urgent cases. Agreement was slight by unweighted kappa (κ = 0.198), fair by quadratically weighted kappa (κw = 0.278), and the prevalence- and bias-adjusted kappa for binary urgent/non-urgent classification was 0.28. Overtriage exceeded 70% for headache and low back pain, and these complaints were the strongest independent predictors of overtriage; higher self-reported pain intensity was strongly associated with overtriage.

Conclusions

The application demonstrated a conservative classification profile with high sensitivity and a low undertriage rate but frequent overtriage that would inflate the apparent urgent caseload if the tool were used unsupervised; it should be considered an adjunct to, rather than a replacement for, professional triage. Findings apply to stable adult walk-in patients whose complaints matched the predefined categories and may not generalize to the broader ED population.