Contextual considerations in summative competency examinations: relevance to the long case.

Long-case patient-based examinations previously formed the basis of summative competency testing in physician certification examinations. These exams were found to be unreliable and have fallen from favor. During the authors' deliberation of the long case in the neurology certification examinations of the Royal College of Physicians and Surgeons of Canada, they considered the examination context and concluded that the appropriate psychometric analysis of the exams is highly contingent on the context. The examination context underlying certification examinations has evolved considerably; within a different context, a more cohesive test system based on a quality assurance framework could better manage substantive psychometric issues around case specificity, comprehensiveness, reliability, and compensability. These arguments are in small part psychometric, but are mostly philosophical and have relevance to the profession and the public.