1. Going Beyond the OSCE for Teaching and Learning Donna H. Kern, MD Alexander Chessman, MD Department of Family Medicine, Medical University of SC Kathryn Chappelle, MA Ryan Palmer, MFA Scott Fields, MD Shawn Blanchard, MD William Toffler, MD Oregon Health & Science University STFM Predoctoral Education Conference 2008

2. The Group Objective Structured Clinical Examination (GOSCE) The OHSU Experience

3. GOSCE Intro & History  1992 As part of a grand Curricular revision  Based on the literature of OSCE  Case based structured patient encounter evaluation  Excellent reviews compared to historical written evaluation.  Found GOSCE to be more reliable in assessing the group and curriculum than the individual “Use of a Group Structured Clinical Examination with first-year medical students,” Elliot, MD, Fields, MD, Keenan, MD, Jaffe, MD, Toffler,MD, Academic Medicine Vol. 69, no 12, Dec. 1994

4. GOSCE Rationale  Emphasis on four main clinical segments Establish rapport/ communication skills Taking an effective history  Pattern recognition  systems and processes Performing physical examination skills Encounter closure  +/- assessment and plan

5. GOSCE Rationale  Resource effective strategy  Direct faculty observation and feedback  Teamwork Emphasis Huddle, Group Interaction, Collective Feedback Group strategy, performance, and debrief Group Grade  Motivation preparation, performance, test intensity

6. GOSCE vs. OSCE Time/ Cost Advantage

7. GOSCE Time/Cost Advantage  Assumptions 120 students 6 cases 12 rooms 20 minutes per encounter 12 SPs

8. GOSCE Time/Cost Comparison OSCE  Time 24 students/day 5 days total  Cost 5 days x 4 hrs=20 hrs $15 SP cost x 20 hrs= $300 $300 dollars x 12 SPs= $3600 total Lunches $150/day= $750 total food cost GOSCE  Time 120 students/day 1 day total  Cost 1day x 4 hrs=4 hrs $15 SP cost x 4 hrs= $60 $60 dollars x 12 SPs= $720 total Lunches $300/day (faculty also get lunch)= $300 total food cost Faculty hours

9. GOSCE Time/Cost Comparison OSCE  Time 24 students/day 5 days total  Cost 5 days x 4 hrs=20 hrs $15 SP cost x 20 hrs= $300 $300 dollars x 12 SPs= $3600 total Lunches $150/day= $750 total food cost GOSCE  Time 120 students/day 1 day total  Cost 1day x 4 hrs=4 hrs $15 SP cost x 4 hrs= $60 $60 dollars x 12 SPs= $720 total Lunches $300/day (faculty also get lunch)= $300 total food cost Faculty hours

10. GOSCE Time/Cost Comparison OSCE Total Costs $4350 GOSCE Total Costs $1020

11. GOSCE Logistics  Student logistics  Faculty logistics  GOSCE procedure

12. Student Logistics  120 Students  24 GOSCE groups  2 GOSCE Rounds Groups 1-12 Round 1 Groups 13-24 Round 2  6 duplicate cases run simultaneously each round

13. Student Logistics  Every student will go at least once. Some will go more than once  1 student will be the “doctor” for the round  Faculty observers give feedback at all stations  Each student receives group, rather than individual, grade

14. Sample GOSCE Day Timeline  Half of groups will take written test 1:00 to 2:30 PM  Half of groups will take GOSCE 1:00 to 2:57 PM GOSCE  Half of groups will take written test 3:30-5:00 PM  Half of groups will take GOSCE 3:07 to 5:04 PM GOSCE Round 1Round 2

17. Faculty Logistics  12 Faculty needed (6 & 6 for duplicate cases)  Faculty give immediate feedback  Faculty, not SP’s, grade students

18. Sample portion of GOSCE checklist

19. GOSCE Logistics: Faculty  Positives Immediate student feedback Truly a “learning” assessment More accurate clinical grading than an SP  Negatives Taxing on faculty time Difficult to “train” graders No “back-up” grader option

20. GOSCE Procedure  2 minutes for strategy  7 minute doctor patient interaction  2 minutes for student “huddle”  2 minute for additions Only 4 are allowed  4 minutes for faculty feedback  3 minutes for transition to next station

21. In Fall of 2007, OHSU upgraded from the paper based inefficiencies of the old…

22. To the sleek, simple digital efficiency of the new!

23. New Method: Clinical Assessment & Learning Center  Utilizes Education Management Solutions (EMS) system: digitally records student test performances indexes video files provides automatic scoring, reporting, and development of quantifiable performance  Faculty can review videos securely and remotely over web

24. Other Features of Center  All grading done on the computer  All GOSCE rooms in one building  Enhanced ability for data tracking  Ability to track individual, group and even instructor ability  Opportunity for post-test video review

25. Clinical Assessment & Learning Center

29. Benefits  Less administrative support  Better ability for student review  Less opportunity for human error  Enhanced data tracking  Better feedback to students  Increased ability to identify remediation needs

30. Continued Challenges  Inter-rater reliability  Increased student review protocols  Integration of curriculum objectives  Grading criteria  Concise feedback  Buyer beware

31. Sample GOSCE Video

32. The Objective Structured Learning Exercise (OSLE) The MUSC Experience

33. The Family Medicine Rural Clerkship  6-week required third-year clerkship  113 Family Medicine practices  Rural/underserved  Population health Community health improvement project Home visits

34. Clerkship Calendar MondayTuesdayWednesdayThursdayFriday 1 1st day at site 2 3 4 5 6 Travel OSCE Project Presentations Academic Days OSLE study Exam

35. Objective Structured Learning Exercise (OSLE)  Required activity for all 3 rd year Family Medicine Rural Clerkship students  Offered at start of the clerkship to prepare students for rotation and final OSCE  WebCT preparation  Groups of 3-6 students  NOT graded (final OSCE is graded)

36. OSLE Stations 6 stations: (1 Faculty member and 1 SP per station) Student Schedule Smoking Cessation and Motivational Interviewing 8:00 am - 8:25 am Cultural competency/CAM 8:30 am – 8:55 am Chronic disease management/ Diabetes 9:00 am – 9:25 am Breaking bad news 9:30 am – 9:55 am Musculoskeletal exam 10:00 am – 10:55 am Cardiac/Pulmonary exam 11:00 am – 11:55 am PDA Exercise 12:00 pm – 12:20 pm

37. Student Satisfaction with the OSLE

38. “The OSLE was one of the most effective learning experiences I have had.” “The OSLE was great! It relieves a lot of stress for the upcoming OSCE since family medicine has potential for encountering such a wide array of problems. It was good to get an idea of what is expected of us and to give us feedback. I wish the other rotations did the same. It was also helpful to have this information in going back to the practices and using the exam techniques we learned in the OSLE on real patients.” “The OSLE was a great experience to receive advice from physicians on how to improve our examination and interviewing techniques. I really benefited from this opportunity for feedback.”

39. Cost for the OSLE  6 faculty members x one half-day (4 hours)  6 trained standardized patients  6 exam/interview rooms  Administrative support for scheduling and handout preparation

40. OSCE (Clinical Exam)  Final day of the clerkship  Standardized patients only- no faculty feedback  Web CT preparation  4 stations + PDA exam  15 % of clerkship grade

41. Cost for the OSCE  8 trained standardized patients  8 exam/interview rooms  Administrative support for scheduling and handout preparation

42. Lessons Learned  Separating teaching and assessment is helpful  Students appreciate the teaching effort  Standardized patient training is critical for an OSLE or OSCE  Faculty enjoy the interaction with students

43. The “Teaching OSCE” The OHSU Experience

44. The Family Medicine Clerkship  5-week required third-year clerkship  16 students / preceptors per rotation  Urban/Suburban (some underserved) --28 hrs patient contact (clinic, hospital, etc.)  Thursdays: didactics / small-group learning

45. OHSU FM Five-Week Clerkship Calendar MondayTuesdayWednesdayThursdayFriday 1 Orientation Morning sessions PMRs 2 Morning sessions OSCE 3 Morning sessions PMRs 4 Morning sessions PMRs 5 Final Exam PMRs Morning Sessions: Common Clinical Problems / Chair Seminars on Family Medicine (BATHE, Smoking Cessation, Asthma, Headaches, Depression, Hypertension, CHF) Patient Management Rounds (PMRs): Student-generated case discussions

46. OHSU “Teaching” OSCE  Required for all third-year students  Offered in 2 nd week of the clerkship  Interactions digitally recorded  Individual students (fully in physician role)  Follow-up activities assigned if learning issues identified  Students assigned self-critiques of interactions  P/NP based on OSCE and subsequent learning activities  Clinical Performance Exam (CPX) end of clerkship year

47. OSCE Scenario List (Six stations, 1 Observer + 1 SP per station) Motivational Interviewing: Smoking Cessation * Bad news: HPV or Lung CA Diabetes exam or history/diagnosis (frequent yeast infections) Back exam or history Insomnia / Alcohol abuse Public Health / Cultural Cases * (meningitis, encephalitis, chemical exposure, lead poisoning, TB) Paper Stations (no SPs: X-Ray, EKG, or Prescription)* * Smoking cessation, one paper station, and one public health/cultural station are always included

48. Individualized & Integrated Teaching Methods Clinical Skills or Knowledge Base: PMRs (present case on related topic) Individual tutoring (practice skills w/ faculty) Preceptors (ask to see pts w/specific issues) Communication or Psychosocial Skills: Video review with faculty Practice with same or similar station Preceptors

49. Student Feedback on Teaching OSCE Takes away stress of performing for a grade; students usually less defensive, more open to feedback and instruction Challenges students to approach each patient in “physician” role Enables students to approach simulated patients more naturally; minimize tendency to “work for checkmarks” Digital recording and self-review generally seen as valuable exercise; good preparation for Board Exams

50. Cost for the Teaching OSCE  4 FM, 1 Public Health faculty x 4 hours  5 trained SPs  6 exam/interview rooms  Video or digital recording ability  Administrative support

51. Lessons Learned at OHSU  OSCEs enable us to evaluate curriculum as well as students: too often we evaluate w/o teaching.  We can better assess some skills when students aren’t working for a checklist  Faculty development should include standard teaching points for observers  Take full advantage of the “teachable moment”!

52. Simulation and the OSLE The MUSC Experience

53. Simulation and the OSLE  Objective Structured Learning Exercise  Normal cardiac and pulmonary exam + “imagination” …  Students miss key components when evaluated in the OSCE: Evaluate PMI 40% Auscultate 4 cardiac areas with patient supine30% Auscultate mitral area with patient lying on left side 60% Examine the neck for jugular venous distention 60% Examine extremities for swelling 50%

54. Simulation and the OSLE  Would simulation improve students’ performance of the cardiopulmonary exam?  Multicenter study in 5 medical schools demonstrated that students trained using Harvey scored higher on physical examination of real patients. Ewy GA. Test of a Cardiology patient simulator with students in fourth-year electives. J Med Educ. 1987 Sep; 62(9) :738-43. Hatala R. Assessing the Relationship between Cardiac Physical Examination Technique and Accurate Bedside Diagnosis during an Objective Structured Clinical Examination. Acad Med. 2007;82(10 Suppl):S26–S29.

55. MUSC Simulation Center

57. The “New” Harvey Cardiopulmonary Patient Simulator Michael S. Gordon Center for Research in Medical Education, Miami, FL  93 lbs (500 lbs lighter)  30 disease states with normal and abnormal breath sounds  “ability to talk”- through a speaker in his head  Costs 40% less ($52K + $4K s/h)  http://www.gcrme.miami.edu/ http://www.gcrme.miami.edu/

58. Cardiopulmonary Exam Station  Standardized patient: normal exam (25 mins)  Students examine Harvey (30 mins) Aortic stenosis: “34 y.o. man presents with syncope.”  Crescendo-decrescendo, grade 3, systolic murmur; S4  Carotid- anacrotic pulse, radiation of murmur  Diminished peripheral pulses  ECG/echocardiography/doppler Aortic valve sclerosis: “70 y.o. man presents for hypertension.”  Crescendo-decrescendo, grade 2, short systolic murmur  No radiation to the carotids  Normal peripheral pulses

59. Video

61. Drivers for using Harvey  Minimal investment start up time faculty development maintenance  Students more engaged in learning  Ability to adapt the scenario to the learner’s needs  Self-directed learning manual for students

62. Barriers to using Harvey  Expense  Lacks portability/students have limited access to Harvey for practicing their skills  Supine--so limited lung exam  Need a multi-stethoscope/speakers for groups  Need for more than one simulator for individual student assessment (OSCE)