Predicting general practice trainee summative fellowship exam performance using commencement-of-training formative assessments: a retrospective cohort study.

BACKGROUND

Fellowship examinations in Australian vocational general practice training assess competency for unsupervised practice. Early identification of trainees at risk of failing these summative examinations facilitates pro-active remediation. This study aimed to explore the utility of formative pre-general practice assessments (PGAs), undertaken at the commencement of training, for predicting final summative examination performance.

METHODS

This was a retrospective cohort study. The PGAs included a written, short-answer, five-case Key Feature Problems test ('PGA-KFP') and an Objective Structured Clinical Examination ('PGA-OSCE'). Multivariable regression was used to examine the relationship between these PGAs and standardized scores on first-attempts at three summative fellowship examinations (an Applied Knowledge Test ('Fellowship-AKT'), a KFP ('Fellowship-KFP') and an OSCE ('Fellowship-OSCE'); and passing all three (versus failing at least one) examination on first attempt. Registrar demographics and training-related factors were included as covariates.

RESULTS

Higher PGA-KFP and PGA-OSCE scores were univariably associated with higher Fellowship-AKT, Fellowship-KFP, and Fellowship-OSCE scores (all p-values ≤ 0.002) and with passing all three examinations on first attempt (p < 0.006). On multivariable analyses, higher PGA-KFP and PGA-OSCE scores were significantly associated with higher Fellowship-AKT, Fellowship-KFP (p values ≤ 0.016) but not Fellowship-OSCE scores. Higher PGA-KFP scores were also predictive of passing all three Fellowship exams on first attempt (p < 0.001).

CONCLUSIONS

This study found intensive formative assessments conducted at the commencement of training to be predictive of final fellowship examination performance in an apprenticeship-based model of general practice training. This suggests utility of these assessments for early-identification of trainees in need of proactive remediation both in Australia and internationally. Further research is required to explore predictive utility of less resource-intensive PGAs.