The peritoneal dialysis orders objective structured clinical examination (OSCE): A formative assessment for nephrology fellows.

BACKGROUND

Peritoneal dialysis (PD) management is a fundamental nephrology skill, especially with the recent emphasis on home dialysis. We report a prospective multicentre cohort study of a formative objective structured clinical examination (OSCE) assessing competence in managing PD-associated bacterial peritonitis, using the unified model of construct validity.

METHODS

The OSCE was developed by the principal investigators and reviewed by two subject matter experts. The test committee (eight nephrologists and one PD nurse) assessed test item difficulty/relevance and determined passing score. There were 22 test items (7 evidence-based/standard-of-care questions). Passing score was 16/22 (73%). No item had median relevance less than 'important', and all were easy to medium difficulty. Content validity index was 0.91. Preliminary validation (16 board-certified volunteers): mean score was 19 ± 2, with 94% (15/16) passing. Kappa = 0.85 [95% confidence interval (CI) 0.77-0.94]. Cronbach's α = 0.70.

RESULTS

Eighty-seven fellows (16 programmes) were tested; 67% passed. Fellows scored significantly less than validators: 17 ± 3 versus 19 ± 2, p < 0.001 [95% CI 1.2-3.6]. Eighty-six per cent of evidence-based/standard-of-care questions were answered correctly by validators versus 54% by fellows; p < 0.001. Ninety-three per cent of fellows recognized that sufficient criteria were present to diagnose peritonitis, but only 17% correctly indicated all three. Seventy-seven per cent recognized peritonitis-associated ultrafiltration failure, but only 17% prescribed 21 days of antibiotic treatment for gram-negative peritonitis. Eighty-five per cent of fellows surveyed agreed/strongly agreed that the OSCE was useful in self-assessing proficiency. Second-year in-training examination and OSCE scores were positively correlated (Pearson's r = 0.57, p < 0.00).

CONCLUSIONS

The OSCE may be used to formatively assess fellow proficiency in managing PD-associated peritonitis.