Supporting examination and assessment using optical mark recognition technology

University of Bristol (2001) J Physiol 536P, S087

Communications: Supporting examination and assessment using optical mark recognition technology

David E. Byrne and Jeremy P.T. Ward

Department of Medical and Dental Education, Guy's, King's and St Thomas' School of Medicine, King's College London, London SE1 9RT, UK

View other abstracts by:


Within the Guy’s, King’s and St Thomas’ (GKT) School of Medicine the annual student intake is now over 360 students each year. Extra demands on staff time and commitment in terms of marking assessments and end-of-term examinations has placed unrealistic workloads on staff, especially in the face of a falling staff/student ratio. Currently, the 1st year medical examination at GKT comprises MCQ, short answer questions (SAQ) and an essay component. Most MCQ examinations used to be marked externally, and used a long-standing format of stem plus five true/false questions, the answers to which were pencilled on standard University of London grids, separate from the question paper. The type and style of question was thus substantially constrained. An increasing amount of assessment now relies on extended MCQs, extending matching formats, and short answers, with a departure from essays. A flexible system was thus required to cater for these varying requirements.

To this end we are currently using a low-priced, commercially available Optical Mark Reader system (Datasimplex: www.datasimplex.co.uk) for in-course and end of year examinations. Fully tailorable examination sheets incorporating graphics and an extremely flexible question format have proved popular with staff and students alike. Students prefer this type of format as their questions and answers are on the same page, and are more interesting. Staff benefits include a great reduction in marking time, and the need for second marking is avoided, resulting in a much faster turn around of exam scripts. The results are also presented ready tabulated on a spreadsheet. There are, however, down sides, including the need for investment in time and money to set up the system, adapting current examination styles and formats to ‘fit’, and identifying persons responsible for the maintenance and upkeep of the associated hardware and software. There is also the philosophical question of whether anything but written answers can fully assess deep learning.

Two-thirds of the 1st year examinations have now moved to this system for MCQ and SAQ examinations, as well as for in-course assessment on a number of modules. We also use the system for course questionnaires and feedback in other years of the medical course. Over 160 000 individual MCQ and SAQ questions were automatically marked for the end of year examinations, with a script rejection and error rate (requiring manual input) of ~2 %. We expect the uptake of the optical mark system to increase to further years of the medical course as the benefits of this system in terms of a cost-effective, efficient and viable method of assessment are proved.



Where applicable, experiments conform with Society ethical requirements.

Site search

Filter

Content Type