1. S tructured O bjective C linical E xamination SOC E

2. S tructured O bjective C linical E xamination

3. What is OSCE ? series of stations with tasks series of stations with tasks planned planned marking form marking form examiner examiner patient : SP patient : SP organization > examination organization > examination

4. Why OSCE ? before OSCE (1975) before OSCE (1975) viva (oral), long case, short case viva (oral), long case, short case  valid ? know how NOT show how know how NOT show how  reliable ? different patients different patients different examiners different examiners

5. Why OSCE ? more valid more valid  show how more reliable more reliable  same task | patient  same examiner or  same structured marking sheet

6. Basic Structure 12348765

7. 12348765 12435687

8. Basic Structure : Parallel 12348765 A B CD

9. Basic Structure : Double 2 123 7654

10. How to start ? blueprint of the whole OSCE blueprint of the whole OSCE design the station design the station design the mark sheet design the mark sheet

11. Blueprint reversed table of classification reversed table of classification

12. Station & Marking learning by doing learning by doing 8 groups : name list 8 groups : name list 8 stations 8 stations 2(p) - 4(x) - 4(d) 2(p) - 4(x) - 4(d) signal signal materials & ID materials & ID task (flexible) task (flexible) lunch : 3rd floor lunch : 3rd floor

13. Station Design station time 4-15 min. station time 4-15 min. total time < 2 hrs total time < 2 hrs focus task focus task examiner used : Y | N, who? examiner used : Y | N, who? pilot pilot

14. Type of Stations static | written static | written practical : technique practical : technique clinical clinical

15. Marking Sheet Design checklist checklist rating scale rating scale score score

16. Checklist dichotomous : Yes | No dichotomous : Yes | No Pros Pros  high objectivity  high reliability  easy to feedback Cons Cons  only quantity check

17. Checklist How to improve? How to improve?  stem clear clear observable observable not too long not too long  overall not too long not too long

18. Rating Scale rating rating quality concern quality concern lower objectivity lower objectivity  lower reliability

19. Rating Scale How to improve? How to improve?  3-7 scale  more clarification of each scale  more raters  rater training common errors of rating scale common errors of rating scale

20. Rating Scale common errors common errors  leniency error  central tendency error  halo effect  logical error  proximity error  contrast error

21. Examiner station developer station developer non station developer non station developer  teacher  not teacher other staff other staff SP SP participation => reliability participation => reliability

22. Observation direct direct indirect indirect  one-way mirror  monitor  video

23. Getting Feedback: How? verbal verbal marked checklist & be the subject marked checklist & be the subject marked checklist & watch video marked checklist & watch video printed answer printed answer relevant papers relevant papers

24. Getting Feedback : When? during the exam during the exam  intra-station  in another station  stress?  NB: too much information! after the exam after the exam  end of all stations

25. Setting an OSCE learning by doing | 8 groups learning by doing | 8 groups structured task | medical student V structured task | medical student V time time  7 min. test (without feedback)  5 min. test + 2 min. with feedback available tools : pls ask available tools : pls ask draft of test and marking sheet : ~ 4 p.m. draft of test and marking sheet : ~ 4 p.m. preparation 8 - 9 a.m. preparation 8 - 9 a.m.

26. Minimal Passing Score criterion-referenced ( อิงเกณฑ์ ) criterion-referenced ( อิงเกณฑ์ )  holistic  modified Angoff norm-referenced ( อิงกลุ่ม ) norm-referenced ( อิงกลุ่ม )  borderline method  relative method

27. Holistic Method medical school’s faculty-wide pass mark medical school’s faculty-wide pass mark e.g. 60% e.g. 60%

28. Modified Angoff Method group of experts group of experts get the OSCE get the OSCE “Think of a group of borderline candidates” “Think of a group of borderline candidates” decide the passing score decide the passing score expert discussion is acceptable in original Angoff expert discussion is acceptable in original Angoff

29. Minimal Passing Score criterion-referenced ( อิงเกณฑ์ ) criterion-referenced ( อิงเกณฑ์ )  holistic  Modified Angoff norm-referenced ( อิงกลุ่ม ) norm-referenced ( อิงกลุ่ม )  borderline method  relative method

30. Borderline Method marking form : checklist + global rating marking form : checklist + global rating all categorized ‘borderline’ students all categorized ‘borderline’ students mean scores of ‘borderline’ group mean scores of ‘borderline’ group

31. Borderline Method Mean borderline score = (72+70+80) / 3 = 74

32. Relative Method 1st method : Wijnen Mothod 1st method : Wijnen Mothod  Passing mark = mean -1.96SE 2nd method 2nd method  60% of the 95th percentile rank score

33. Minimal Passing Stations criterion-referenced criterion-referenced

34. Staff & OSCE : Like emotional comfort emotional comfort validly assess validly assess consistent consistent

35. Staff & OSCE : Dislike too compartmentalized too compartmentalized no opportunity to observe the complete patient evaluation of the student no opportunity to observe the complete patient evaluation of the student repetitive nature => boring repetitive nature => boring

36. Students & OSCE fairer than other methods fairer than other methods less stressful less stressful unsure whether the important aspects tested unsure whether the important aspects tested

37. Limitations of OSCE lengthy preparation lengthy preparation need more observational skill of the staff need more observational skill of the staff costly costly low inter-station correlation low inter-station correlation test security? test security?

38. What’s next? evaluation => learning evaluation => learning summative => formative summative => formative

39. Innovation senior student as SP and examiner in OSCE senior student as SP and examiner in OSCE study sheet listing Dx that might appear on the OSCEs study sheet listing Dx that might appear on the OSCEs add structured oral exam into OSCE add structured oral exam into OSCE GOSCE GOSCE

40. GOSCE : Group OSCE Pros Pros  economy  mutual teaching  mutual support  opportunity to examine social skill Cons Cons  lack of individual assessment  different participants do different tasks

41. Potential Use of GOSCE formative assessment formative assessment end-of-course assessment end-of-course assessment exploring interpersonal relationship exploring interpersonal relationship teaching method for short course teaching method for short course

42. Re-using OSCE stations across rotation in the same academic year across rotation in the same academic year  statistically OK from year to year from year to year  statistically not OK

43. Conclusion : OSCE What ? What ?  stations + tasks + checklist Why ? Why ?  more valid, more reliable How ? How ?  How to organize?  How to analyze?