The use of OSCE to assess Patient Care, Professionalism and Interpersonal Communication Milestones in EM residents Miriam Kulkarni, MD, Harsh Sule, MD, MPP, Jill Ripper, MD and Tiffany Murano, MD Dept. of Emergency Medicine, Rutgers New Jersey Medical School, Newark, NJ Milestones Assessment Clinical Setting Evaluations Some milestones difficult to assess Faculty overestimate resident achievement Simulation SDOT Challenging in clinical setting Not based on milestones Limited by types of patients in the ED

The use of OSCE to assess Patient Care, Professionalism and Interpersonal Communication Milestones in EM residents Miriam Kulkarni, MD, Harsh Sule, MD, MPP, Jill Ripper, MD and Tiffany Murano, MD Dept. of Emergency Medicine, Rutgers New Jersey Medical School, Newark, NJ Milestones Assessment Clinical Setting Evaluations Some milestones difficult to assess Faculty overestimate resident achievement Simulation SDOT Challenging in clinical setting Not based on milestones Limited by types of patients in the ED

The use of OSCE to assess Patient Care, Professionalism and Interpersonal Communication Milestones in EM residents Miriam Kulkarni, MD, Harsh Sule, MD, MPP, Jill Ripper, MD and Tiffany Murano, MD Dept. of Emergency Medicine, Rutgers New Jersey Medical School, Newark, NJ Milestones Assessment Clinical Setting Simulation OSCE Difficult to assess milestones Immediate, direct feedback Incoming first-year resident baseline assessment Assess resident performance and track progress SDOT Challenging in clinical setting Not based on milestones Limited by types of patients in the ED Evaluations Some milestones difficult to assess Faculty overestimate resident achievement

The use of OSCE to assess Patient Care, Professionalism and Interpersonal Communication Milestones in EM residents Miriam Kulkarni, MD, Harsh Sule, MD, MPP, Jill Ripper, MD and Tiffany Murano, MD Dept. of Emergency Medicine, Rutgers New Jersey Medical School, Newark, NJ BACKGROUND OBJECTIVES Four 20-minute OSCE scenarios were developed by a group of EM educators One scenario was developed for each EM training year and included a checklist based on EM Milestones The OSCE was administered to residents by a faculty observer The patients were played by standardized patients (SPs) Consultants were played by a faculty member or a senior resident At the end of the scenario, there was a five minute period for feedback on performance of the EM Milestones and overall clinical performance, which was provided by the faculty member. The SP and faculty member provided feedback on patient communication and professionalism Difficult to evaluate all Emergency Medicine (EM) milestones in the clinical setting Faculty tend to overestimate resident achievement The Standardized Direct Observation Tool (SDOT) is reliable in evaluating clinical performance…but: -Can be challenging to administer in the clinical setting -Is not based on milestones -The number of milestones extrapolated is limited by types of patients in the ED There is a need for an evaluation tool based on the EM Milestones which would create a standardized experience for each resident EM YearEM milestones/Level addressed 1Milestones 1, 2, 3 and 4 (level 1) 2Milestones 1, 2, 3, 4, 5, 16, 18 and 19 (level 2) 3Milestones 1, 2, 3, 4, 5 and 7 (level 3) 4Milestones 2, 16, 20, 21, 22 1.Evaluate residents’ performance of EM Milestones in a standardized format 2.Provide residents with feedback on their performance of EM Milestones. 3.Provide residents with feedback on their communication skills and professionalism METHODS

1.ACGME and ABEM. The Emergency Medicine Milestone Project. December Available from: docs/default-source/migrated-documents-and-files/em-milestones. pdf?sfvrsn 4. Accessed February Wallenstein J, Ander D. Objective structured clinical examinations provide valid clinical skills assessment in emergency medicine education. West J Emerg Med Jan;16(1): Reliability of a core competency checklist assessment in the emergency department: the Standardized Direct Observation Assessment Tool. Shayne P, Gallahue F, Rinnert S, Anderson CL, Hern G, Katz E; CORD SDOT Study Group. Acad Emerg Med Jul;13(7): Real-time inter-rater reliability of the Council of Emergency Medicine residency directors standardized direct observation assessment tool. LaMantia J, Kane B, Yarris L, Tadros A, Ward MF, Lesser M, Shayne P; SDOT Study Group II. Acad Emerg Med Dec;16 Suppl 2:S51-7. REFERENCES CONCLUSIONS The use of OSCE to assess Patient Care, Professionalism and Interpersonal Communication Milestones in EM residents Miriam Kulkarni, MD, Harsh Sule, MD, MPP, Jill Ripper, MD and Tiffany Murano, MD Dept. of Emergency Medicine, Rutgers New Jersey Medical School, Newark, NJ This educational innovation allowed the administration of a standardized patient encounter with a Milestone-based evaluation The OSCE allowed assessment of the Milestones that are more difficult to evaluate in the clinical setting, such as professionalism and interpersonal communication skills Immediate direct feedback by the standardized patient was invaluable and well received by the residents The OSCE provided an invaluable baseline assessment of incoming first-year resident performance, with the potential to correlate to Core Entrustable Professional Activities that a graduating medical student is ideally able to perform The OSCE provided valuable milestone-specific and general information regarding resident performance and may be used to track resident progress Milestone 1Emergency Stabilization (PC1) 2Performance of Focused History and Physical Exam (PC2) 3Diagnostic Studies (PC3) 4 (level 1) Diagnosis (PC4) Constructs a list of potential diagnoses based on chief complaint and initial assessment 5Pharmacotherapy (PC5) 7 (level 3) Disposition (PC7) Formulates and provides patient education regarding diagnosis, treatment plan, medication review and PCP/consultant appointments for complicated patients Involves appropriate resources (e.g., PCP, consultants, social work, PT/OT, financial aid, care coordinators) in a timely manner Makes correct decision regarding admission or discharge of patients Correctly assigns admitted patients to an appropriate level of care (ICU/Telemetry/Floor/ Observation Unit) 16Professional Values (PROF1) 18Patient Centered Communication (ICS1) 19Team Management (ICS2) 20Practice-based Performance Improvement (PBLI) 21Patient Safety (SBP1) 22Systems-based Management (SBP2)