Where Are We and Where Are We Going? A GME Program Scorecard for Continuous Programmatic Improvement.

BACKGROUND

Graduate medical education (GME) leaders face challenges in evaluating program performance due to fragmented data systems and limited resources. Dashboards have been proposed, but few are scalable, participant-informed, and feasible without new infrastructure.

OBJECTIVE

To develop and implement a centralized, participant-informed GME Program Scorecard (GPS) that enables continuous program evaluation and improvement using existing data systems without new data entry.

METHODS

The GPS was developed at a large academic medical center in 2022-2023. A consensus development panel of 20 GME administrators identified 42 candidate metrics across 13 systems. Program directors (PDs) from the largest residency programs were surveyed (11; 9 responses, 81.8%) to rate each metric's value. The 17 most supported metrics were integrated into a Tableau-based dashboard. The GPS was piloted with 4 programs (3 Accreditation Council for Graduate Medical Education [ACGME]-accredited, 1 non-ACGME-accredited), involving 8 PDs and program coordinators. Feedback was collected via structured forms and a focus group. Metrics were rated on a 1 to 5 scale, with lower scores indicating higher priority.

RESULTS

Pre-intervention survey results guided prioritization, with 92.9% of metrics receiving a mean score <3. Pilot users reported high acceptability and identified design refinements incorporated before institutional launch. Development required 2920 analyst hours and $1,600 in licenses. Since launching in January 2024, 309 users (61.4% of 503 potential users) have accessed the GPS with 3933 total views.

CONCLUSIONS

The GPS demonstrated feasibility, requiring 2920 analyst hours and $1,600 in licenses, and with 61.4% of potential users accessing the dashboard within 22 months of launch.