Evaluation of structured oral examination format used in the assessment of undergraduate medical course (MBBS) of the University of Dhaka

Abstract

Objectives of this cross sectional descriptive study was to evaluate critically the current status of structured oral examination (SOE) format as practiced in the professional examination of undergraduate medical course (MBBS) and views of the faculties regarding the concept of SOE as an assessment tool. The study was conducted in 9 medical college examination centers of Dhaka University in July 2007. There were 36 examiners in 18 SOE board, 26 of them were interviewed with a semi-structured questionnaire and SOE boards were observed with a checklist. A total of 2455 questions used in SOE to assess 123 students, were recorded and analyzed using another checklist. These questions were used to assess learning hierarchy and content coverage using forensic medicine as a reference subject. Analysis of the questions revealed majority (97%) were of recall type, only 3% were interpretation and problem solving types. The questions for 119 (97%) examinee did not address 10%-50% content area. About 38% examiners responded that they had no clear idea regarding learning objectives and none had idea regarding test blueprint.The examiners marked the domain of learning measured by SOE in favor of cognitive skill (61%), communication skill (38.5%), motor skill (11.5%), behavior and attitude (19%). No examiner prepared model answer of SOE questions by consensus with other examiner. Though more than 80% examiner agreed with the statement that pre-selection of accepted model answer is an important element for success of SOE. But no examiners of any SOE boards practiced it. Similarly, none of the examiners of SOE board kept records of individual question and the answer of the examinees. No boards maintained equal time for a candidate during SOE by using timer or stop watch. Examiners of 8 boards (44%) did not use recommended rating scale to score individual response of examinee rather scored in traditional consolidated way at the end of the candidates examination. Majority (94%) boards scored the prompted answer and allowed another questions when a candidate failed to answer. During SOE conduction, 22% examiner were absent from the board for a prolonged period and 3% was engaged in marking the written scripts. About 56% of the examiners arrived late than schedule time. Behaviors of 14% examiner showed abusing to the candidates. The study revealed that the objectives of introducing SOE as assessment tool in undergraduate medical curriculum was not achieved and it was not appropriately implemented. The various elements of SOE were not followed in most of the sessions of examinations. However, the reasons for not implementing vis a vis following the attributes of SOE were not explored. The study was done only in forensic medicine but similar situations may exist in other subjects. It is recommended that further study may be instituted to determine the causes of not achieving the objective of SOE in undergraduate medical evaluation system. The examiners should be motivated and trained up adequately to implement the elements of SOE successfully as valid, reliable and objective assessment tool.

Ibrahim Med. Coll. J. 2015; 9(1): 1-10