Whilst no method of standard setting is without limitations, the use of the modified Angoff method appears to be favoured by many medical courses when establishing the cut-off score (or pass mark) for clinical MCQs. This method requires a panel of judges to define a borderline group; that is, a group with as equal a chance of passing as of failing the specific assessment and to then determine, independently of each other, the percentage of the group that would be expected to pass each of the questions in the assessment. Following subsequent discussion between judges, the cut-off score is established as the totalled average of each judge’s final score for every question and thus an absolute standard is set. Whilst conceptually simple and relatively easy to perform, the test may be particularly suitable for MCQ exam formats and for those assessments that gauge a readily agreed-upon competency. In addition, an acceptable minimum number of judges of around 8-10 may also preclude full adoption of such methodology in many Departments where staff numbers and/or subject knowledge is relatively low. Might therefore such a method be unsuitable for short answer questions (SAQ) in physiology where both the format and the required competency(ies) be less amenable to simple judgment? Or are staff numbers a greater issue that might limit adoption? This short, interactive session will utilise past paper exam questions from physiology exams undertaken by students at the University of Birmingham to determine an Angoff score for each question prior to revealing and comparing against actual performance statistics. The session will conclude with a discussion of the reliability of the modified Angoff for standard setting SAQs and the relative merits of this and other methods in setting the standard of exams where competency may be less easily defined.
Physiology 2012 (Edinburgh) (2012) Proc Physiol Soc 27, SA74
Research Symposium: Can we use the modified Angoff for short answer questions?
P. Kumar1
1. College of Medical and Dental Sciences, School of Clinical and Experimental Medicine, Birmingham, United Kingdom.
View other abstracts by:
Where applicable, experiments conform with Society ethical requirements.