Performance of a consensus scoring algorithm for assessing pediatric advanced life support competency using a computer screen-based simulator.

OBJECTIVE

To develop a computer screen-based simulator that may be used as a pediatric advanced life support (PALS) cognitive skill assessment tool and to pilot test a consensus-based scoring system for the simulator.

DESIGN

Development of an evaluation tool, followed by prospective, observational study of tool performance.

SETTING

Tertiary care pediatric hospital.

SUBJECTS: A total of 100 PALS providers from multiple disciplines.

INTERVENTIONS: Using a consensus process with a group of six experts in pediatric emergency and critical care medicine, we developed scoring algorithms to measure performance on four interactive PALS scenarios (supraventricular tachycardia, pulseless electrical activity, ventricular fibrillation, and bradycardia). PALS providers (n = 100) completed the scenarios on the simulator and the computer assessed their performance using the scoring algorithm.

MEASUREMENTS AND MAIN

RESULTS

Case management scoring audits agreed 100% with computer scoring during pilot testing, indicating excellent reliability. The mean time to complete all four cases was 13.8 mins. Performance scores were highest for supraventricular tachycardia management and lowest for pulseless electrical activity management. Survival was significantly more common than death in the supraventricular tachycardia and ventricular fibrillation scenarios, whereas death was more common in the pulseless electrical activity scenario (p < .004). Physician status predicted a higher aggregate score as well as higher scores in the supraventricular tachycardia (p < .001), pulseless electrical activity (p = .041), and bradycardia (p = .006) scenarios. Participants who completed the PALS course on the same day as their assessment scored higher on the supraventricular tachycardia scenario (p = .041).

CONCLUSIONS

Personal computer-based simulation can be used to evaluate performance against consensus criteria in a large number of PALS providers. This technology could supplement traditional curricula by facilitating frequent knowledge assessments as part of a PALS competency maintenance regimen.