CJEM · 2026
The impact of faculty development interventions on supervisor narrative feedback quality in competency-based medical education.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Woods Rob, Spadafore Max, Thoma Brent, Martin Lynsey, Chan Teresa, Bandi Venkat
Аннотация
OBJECTIVES: Improving the feedback that supervisors provide trainees in medical education is challenging. The development of automated methods of rating the quality of narrative feedback has created an opportunity to quantify the impact of interventions focused on this goal.
METHODS: We used a previously developed natural language processing model based on the Quality Assessment of Learning (QuAL) score to rate the quality of each comment in our Royal College Emergency Medicine specialty training program from July 2018 to June 2024. We quantified the effectiveness of three faculty development interventions (e-mail feedback, audit and feedback workshops, and assessment form modifications) by comparing pre- and post-intervention QuAL scores.
RESULTS: During the study period 9709 narrative comments were scored. The average QuAL score rose from 3.2/5 in 2019 to 4.5/5 in 2024. All interventions resulted in an overall improvement in the mean QuAL score as measured by assessor weighted mean: individual e-mail with suggestions for improvement (35.1%), assessment form template change (24.3%), workshop (19.1%) and individual e-mail thanking them for their high-quality comments (11.0%).
CONCLUSIONS: By combining a multi-faceted faculty development approach with a scalable measurement of our outcome, we have demonstrated an improvement in supervisor narrative comment quality and quantified the impact of each intervention. While it is likely that the interventions were complementary, individual e-mails to faculty with suggestions for improvement and modifications to the assessment form were associated with largest improvements in quality. These results may guide to other programs seeking to improve the quality of supervisor narrative comments.
METHODS: We used a previously developed natural language processing model based on the Quality Assessment of Learning (QuAL) score to rate the quality of each comment in our Royal College Emergency Medicine specialty training program from July 2018 to June 2024. We quantified the effectiveness of three faculty development interventions (e-mail feedback, audit and feedback workshops, and assessment form modifications) by comparing pre- and post-intervention QuAL scores.
RESULTS: During the study period 9709 narrative comments were scored. The average QuAL score rose from 3.2/5 in 2019 to 4.5/5 in 2024. All interventions resulted in an overall improvement in the mean QuAL score as measured by assessor weighted mean: individual e-mail with suggestions for improvement (35.1%), assessment form template change (24.3%), workshop (19.1%) and individual e-mail thanking them for their high-quality comments (11.0%).
CONCLUSIONS: By combining a multi-faceted faculty development approach with a scalable measurement of our outcome, we have demonstrated an improvement in supervisor narrative comment quality and quantified the impact of each intervention. While it is likely that the interventions were complementary, individual e-mails to faculty with suggestions for improvement and modifications to the assessment form were associated with largest improvements in quality. These results may guide to other programs seeking to improve the quality of supervisor narrative comments.