Dr Matt Homer publishes paper on Objected Structured Clinical Examinations
Dr Matt Homer has published a new paper about the impact of assessor variation on candidate pass/fail decisions in Objected Structured Clinical Examinations (OSCEs).
Dr Homer’s paper uses a large and unique General Medical Examination database to confirm the very strong impact of examiners on assessments of clinical performance (commonly known as OSCEs – Objected Structured Clinical Examinations).
For the first time, the work shows that examiners tend to be similarly stringent in the their marking between different types of scoring – one based on assessing key domains of performance (e.g. interpersonal skills and patient management), and the other a holistic overall grade. It also evidences a potential bias in the process of setting of examination standards – caused by the variation in scoring by examiners (this effect often referred to informally as that of ‘hawks and doves’).
The key messages are that OSCE-type exams must have a sufficient number of stations to be psychometrically defensible, and that more work is needed, ideally in different contexts, to assess the impact of examiners on scores, standards and, ultimately, pass/fail decisions in high stakes settings.
