Download presentation
Presentation is loading. Please wait.
Published byTamsyn Hodges Modified over 6 years ago
1
Structured Oral Examination as an assessment tool
Dr Samira Rahat Afroze Internal Medicine Dept IMC & BIRDEM General Hospital
2
What will be covered Introduction to oral examination
Structured oral examination (SOE) Oral examinations : Bangladesh perspective The process of conducting SOE Do’s & Don’ts
3
The oral examination / viva A traditional form of assessment
Face-to-face discussion One or more examiners ask the candidate questions Allows assessment of intellectual abilities professional attitude communication skill appearance confidence Cox KR, Ewan CE. The Medical teacher. Edinburgh : New York: Churchill Livingstone; 1982.
4
Where applied in medical field?
Undergraduate medical education Postgraduate medical education Job interview Today…. I shall be focusing on undergraduate oral examinations
5
Blooms Taxonomy 3 Domains of Education
6
BLOOM’S TAXONOMY Oral examination is aimed at assessing mainly the higher degree of cognitive and problem solving skills of the candidates
7
Educational Handbook for Health Personnel (WHO)
Advantages Direct personal contact with candidates Allow candidate to formulate his own replies without cues Opportunity to take mitigating circumstances into account Candidate can be asked how he arrived at an answer depth of knowledge Flexibility in moving to & fro a candidate’s strong points to weak points Simultaneous assessment by ≥ 2 examiners Cox, 1982 ; Gibbs et al., 1993 ;Gibbs et al., 1988; Wakeford et al., 1995; Jolly & Grant, 1997; Wass et al., 2003
8
Lack of standardization; objectivity & reproducibility of results
Disadvantage (WHO) Lack of standardization; objectivity & reproducibility of results Shortage of trained examiners Costly in terms of professional time Difficulty in sampling from curriculum Costly in terms of professional time in relation to the limited value of information yielded. better tested in written examinations. Largely test factual knowledge Permit favoritism & biasness Suffer from undue influence of irrelevant factors
9
Traditional oral exam (TOE)
Subjective test Fails to assess the cognitive and problem solving skills properly The atmosphere of the examination is often threatening Independent marking by a pair of examiners is rarely done
10
Poor Reliability ? Validity
Different assessment of different content areas Different difficulty levels of the questions asked Varying levels of prompting or help provided ? Validity
11
Structured oral examination
And… This is where stuctured oral examination comes in
12
“If the decision is taken to retain the orals in the examination system, there are several suggestions regarding best practice that should be followed.”
13
Structure the oral on clinical scenarios
with well-defined learning outcomes 2) Use a number of orals “reliability when using a number of orals is better than the reliability of a single oral examination” Use a number of examiners Careful selection of examiner and paring a new examiner with a more experienced person.
14
4) Ask all candidates the same questions
improve sampling of the syllabus avoid overlap between the orals and other components of the assessment Care should be taken to
15
Use descriptors, rubrics or criteria for answers
Pre-planned blue print provide clear written guidelines about asking questions what is & is not an acceptable answer Rating scale must be standardized
16
Test Matrix CVS Resp Neuro Nephro HBS GIT Endo Psy Derm Knowledge
(Define) x Comprehension (Interpretation) Analysis (Differentiate) Synthesis (problem solving) Evaluation (Criticize / defend) Define heart failure Interpret : Chest pain & SOB, ECG & CXR findings Analysis : How will you clinically differentiate from Acute severe asthma? Synthesis: What steps will you take to manage this patient? Evaluation: SOB is not responding to treatment, how will you evaluate?
17
6) Train the examiners All examiners must be familiar with
the rating scale.
18
Bangladesh perspective
19
Written examination format SAQ & MCQ
Undergraduate medical curriculum of extensive modification of the assessment system Written examination format SAQ & MCQ with 10% mark added by formative assessment. Traditional practical & oral assessment OSPE/OSCE and SOE modified to
20
MBBS professional examinations: SOE implemented
Pre-clinical : Jan, 2005 Para-clinical :Jan, 2007 Clinical : July, 2009
21
SOE (curriculum recommendations) Pre-examination meeting of examiners Question of varying degrees of difficulty recall 50-60% interpretative 20% problem solving questions 10-20% Correct answer & rating scale Decisions made jointly Questions should be typed on a card help to neutralize the assessor effect to some extent
22
Procedure Cordial atmosphere Two boards : 2+2 = 4 examiners
Equal time : 15 minutes Timer or stop watch Candidate pick up the cards randomly from a box / boxes of defined domain After answering the questions the examiner place a tick in an appropriate box on a prepared rating scale candidates feel secured and tension free environment
23
How IS it PRACTISED ? Research Ibrahim Med. Coll. J. 2015; 9(1): 1-10
To evaluate critically the current status of structured oral examination (SOE) format as practiced in the professional MBBS examination Views of the faculties regarding the concept of SOE as an assessment tool.
24
SOE as assessment tool in undergraduate medical curriculum is not achieved & it is not appropriately implemented
25
How to improve? Policy makers : regular and intensive training program
Continuous studies determine the reason(s) its suitability in our medical curriculum redesigning the SOE as an assessment tool For faculty development
26
The other Do’s
27
Steps of Oral Examination
Phrase question clearly. Allow sec…… Encourage student to respond. Acknowledge correct answer. Ask lower to higher cognitive order questions.
28
If still fails to answer …. Shift to the next question
A student often cannot understand the question. Is the question clear to you? Do you want me to rephrase the question? Which part you did not understand? Is it too difficult? Studies have shown that even modest increase of wait time, the correctness improves. Allow time If still fails to answer …. Shift to the next question
29
Feedback Always an important aspect of upgrading validity & reliability of any assessment tool. Student : better if given individually Provides opportunity to find out the gaps in learning & teaching
30
What examiners should avoid during oral exams
31
interrupt provide answer criticize
32
Conclusion Structured oral examinations can be a useful component of assessment It has a special place in medical education because……. it assess the communication skill of the candidate Competency of a doctor is also judged by way he/she talks
35
Give an example
37
Weingarten MA, Polliack MR, Tabenkin H, et al. 2000
Weingarten MA, Polliack MR, Tabenkin H, et al Variations among examiners in family medicine residency board oral examinations. Med Educ. 34:13–17 best practice-to-date oral examinations have been criticized on the basis that inter-observer agreement does not necessarily imply reliability because two examiners sitting together during an examination inevitably affect each others assessment leading to a tendency towards agreement
38
Spike N, Jolly B. 2003. Are orals worth talking about
Spike N, Jolly B Are orals worth talking about? Medical Education. 37: 92–93. 1. there was potential for adverse educational impact on the candidates; 2. there was no allowance for compensation and; 3. it could be unsuitable for borderline candidates.
39
Is there a need of oral examinations?
40
If we select MCQs to assess knowledge (knows) Extended matching items or varieties of short-answer questions to assess application of knowledge (knows how) OSCE to assess ‘‘shows how’’ direct observation of practice to assess ‘‘does’’ Davis MH, Karunathilake I The place of the oral examination in today’s assessment systems. Medical Teacher. 27(4):
41
Giving up such a tradition is not easy And
It is likely that the oral examination will continue to have a place in assessment systems for some time to come. It has been decided that….. If structured properly, Oral examinations can be a useful assessment
42
One example I found online ………. was on Post-graduate oral exam
43
Verbal scores were converted to numerical scores,
Outstanding = 9 Excellent = 8 Good = 7 Satisfactory = 6 Bare pass = 5 Not adequate = 4 Unsatisfactory = 3 Poor = 2 Dangerous = 1
44
Descriptive study, 2007, 9 centers
36 examiners in 18 SOE boards, 26 interviewed SOE boards were observed with a checklist Analysis of the questions 97% recall type 3% interpretation & problem solving types
45
38% had no clear idea regarding learning objectives
Examiners response 38% had no clear idea regarding learning objectives 100% regarding test blueprint Model answer of SOE questions was not prepared Observation 56% arrived late 22% absent for prolonged period examiners 3% marking the written scripts 14% showed abusive behaviors 44% rating scale not used 94% boards scored prompted answer Equal time: not maintained 80% examiners agreed there should be prior preparation rather scored in traditional way at the end
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.