1. Structured Oral Examination
Dr Samira Rahat Afroze
Registrar
Internal Medicine Dept
IMC & BIRDEM General Hospital

2. The oral examination or viva is a traditional form of assessment in which one or more examiners ask the candidate questions. Allows assessment of appearance, manner, personality alertness, confidence, honesty and self-awareness. Face-to face discussions allows personal characteristics and intellectual abilities to be explored to a degree unavailable to other forms of examination.

3. Blooms Taxonomy
3 Domains of Education

5. Educational Handbook for Health Personnel (WHO; 1998)
Advantage
Direct personal contact with candidates
Allow candidate to formulate his own replies without cues
Opportunity to take mitigating circumstances into account
Candidate can be asked how he arrived at an answer
Flexibility in moving from candidate’s strong points to weak points
Simultaneous assessment by 2 examiners

6. Disadvantage
Lack of standardization
Suffer from undue influence of irrevalent factors
Lack objectivity & reproducibility of results
Shortage of trained examiners
Permit favoritism & possible abuse of the personal contact
Costly in terms of professional time in relation to the limited value of information yielded.

7. direct personal contact (Cox, 1982)
assessing problem-solving and reasoning
(Sandars, 1998; Wass et al., 2003)
recognition of safe and competent clinicians
(Zelenock et al., 1985)
assessing professionalism and ethics (Wass et al., 2003)
opportunity to probe depth of knowledge
(Cox, 1982; Gibbs et al., 1988; Jolly & Grant, 1997)
flexibility in moving from one area to another
(Deale,1975; Schwartz & Sein, 1987; Gibbs et al., 1998; Wakeford et al., 1995)
feedback on curriculum (Colton & Peterson, 1967)
the ability to tailor the questions asked to the needs of each individual candidate (Gibbs et al., 1993)

8. disadv Difficulty in sampling from a wide area
largely test factual knowledge, which can
be better tested in written examinations.

9. Poor Reliability Poor Validity
Different assessment of different content areas
Different difficulty levels of the questions asked
Varying levels of prompting or help provided
Poor Validity

10. Is there a need to use oral examinations?
If we select MCQs to assess knowledge (knows) Extended matching items or varieties of short-answer questions to Assess application of knowledge (knows how) OSCE to assess ‘‘shows how’’ direct observation of practice to assess ‘‘does’’
Davis MH, Karunathilake I The place of the oral examination in today’s assessment systems. Medical Teacher. 27(4):

11. Giving up such a tradition is not easy
And
It is likely that the oral examination will continue to have a place in assessment systems for some time to come.

12. “If the decision is taken to retain the orals in the examination system, there are several suggestions regarding best practice that should be followed.”

13. 1) Structure the oral on clinical scenarios
Based on a clinical case with well-defined learning outcomes
Can assess
knowledge, interpretive ability, problem solving & attitudes

14. 2) Use a number of orals
“reliability when using a number of orals is better than the reliability of a single oral examination”

15. 3) Use a number of examiners
to use multiple vivas, with a single examiner per viva
final evaluation on independent assessments by multiple examiners

16. 4) Ask all candidates the same questions
improve sampling of the syllabus
elimination of overlap between the orals and other
components of the assessment.

17. 5) Use descriptors, rubrics or criteria for answers
provide clear guidelines on what is & is not an acceptable answer
Pre-planned blue print
Rating scale must be standardized

18. 6) Train the examiners
Crucial
All examiners must be familiar with the rating scale.

20. Verbal scores were converted to numerical scores,
Outstanding = 9
Excellent = 8
Good = 7
Satisfactory = 6
Bare pass = 5
Not adequate = 4
Unsatisfactory = 3
Poor = 2
Dangerous = 1

21. Oral examinations in MBBS abroad
There has been a move away from oral examinations at postgraduate level in Australia and Canada & at a medical undergraduate level in the UK.
India?
Pakistan?
Malaysia?

22. Bangladesh perspective
SOE as assessment tool in undergraduate medical curriculum is not achieved and is not appropriately implemented.

23. Research
Ibrahim Med. Coll. J. 2015; 9(1): 1-10

24. Descriptive study
To evaluate critically the current status of
structured oral examination (SOE) format as practiced in the professional MBBS examination
Views of the faculties regarding the concept of SOE as an assessment tool.

25. 9 medical college examination centers of Dhaka University in July 2007
36 examiners in 18 SOE board, 26 of them were interviewed with a semi-structured questionnaire and SOE boards were observed with a checklist.
A total of 2455 questions used in SOE
to assess 123 students,

26. Results Analysis of the questions 97% recall type
3% interpretation & problem solving types
In 97% examinee did not address 10%-50% content area.

27. 38% had no clear idea regarding learning objectives
Examiners response
38% had no clear idea regarding learning objectives
& none had idea regarding test blueprint
Domains of learning cognitive skill (61%)
communication skill (38.5%)
motor skill (11.5%)
behavior & attitude (19%)
Model answer of SOE questions was not prepared
Although 80% examiner agreed that pre-selection of model answer is important

28. No boards maintained equal time for a candidate
44% examiners did not use recommended rating scale, rather scored in traditional way at the end
94% boards scored the prompted answer and allowed another questions when a candidate failed to answer.

29. During SOE conduction
22% examiner were absent from the board for a prolonged period
3% was engaged in marking the written scripts.
56% of the examiners arrived late than schedule time.
Behaviors of 14% examiners showed abusing to the candidates.

30. What can we do to help?
The policy makers must take urgent necessary action to arrange regular and intensive training program for faculty development.
Further study may be undertaken to determine the reason(s) of not being appropriately
implementing SOE and its suitability in our medical curriculum.
Also, examiners should rethink for redesigning the SOE as an assessment tool.

31. What examiners should avoid during oral exams

34. Weingarten MA, Polliack MR, Tabenkin H, et al. 2000
Weingarten MA, Polliack MR, Tabenkin H, et al Variations among examiners in family medicine residency board oral examinations. Med Educ. 34:13–17
best practice-to-date oral examinations have been criticized on the basis that inter-observer agreement does not necessarily imply reliability because two examiners sitting together during an examination inevitably affect each others assessment leading to a tendency towards agreement

35. Spike N, Jolly B. 2003. Are orals worth talking about
Spike N, Jolly B Are orals worth talking about? Medical Education. 37: 92–93.
1. there was potential for adverse educational impact on the candidates; 2. there was no allowance for compensation and; 3. it could be unsuitable for borderline candidates.