1. Educational Video-based Scenarios and Evaluative Instrument For Handoff Communication Saba Berhie 1 ; Vineet M Arora 1 ; Leora Horwitz 2 ; Mark Saathoff 1 ; Paul Staisiunas 1 ; Jeanne M. Farnan 1 1 University of Chicago, Chicago, IL; 2 Yale University, New Haven,CT Background Single institution and small sample size Future work planned at partner institutions With generous funding from: AHRQ R03 Grant # 1R03HS018278-01 NIA T35 Grant #5T35AG029795-02 Instrument Development: Handoff CEX CommunicationProfessionalismSetting Superior “Check post- transfusion HgB at 2pm. If <10, transfuse 2 U“ No inappropriate comments, not hurried Satisfactory “the only to do is check CBC at 2pm” “I’ve had a rough day, lets be quick. I’ll start with the sickest first, she’s a train wreck” Un-satisfactory “Oh yeah, can you check labs?” “the ER could not intubate to save their life” Research Funding Limitations Workshop Objectives & Structure Conclusions Utilize videos which highlight varying levels of handoff performance. Rate and assess each video utilizing Handoff CEX instrument WORKSHOP OUTLINE Two 60-minutes workshops offered Six videos watched and rated independently by faculty participants using Handoff CEX Post-workshop roundtable held after viewing the videos 3-item Handoff CEX shows promise as a reliable and valid tool to assess varying levels of videos depicting handoff performance Raters had difficulty distinguishing between satisfactory and superior levels Correlation between communication and professionalism Future work in real-time clinical scenarios Results 14 faculty from 2 departments participated 73 of a possible 90 (82%) handoff observations were captured Reliability Cronbach’s alpha: 0.81 Kendall’s coefficient of concordance: 0.59 Based on “Mini- CEX” instrument widely used in internal medicine (Norcini,2003) Peer Evaluation Pilot Testing Internal Medicine residents at the University of Chicago: 366 handoff observations Factor analysis revealed that three domains captured 82% of the variance of overall sign-out quality Communication, Professionalism and Setting 0 2 4 6 8 10 Unsatisfactory SatisfactorySuperior Communication p <0.001 Validity Test of trend across ordered groups: p<0.001 Two-way ANOVA No rater bias; results significant only for difference between “superior” & “unsatisfactory” levels of performance July 2011– ACGME sets new limits for resident duty hours Unintended consequence is increase in number of handoffs Few trainees receive education on effective handoff communication (Horwitz, 2006) Lack of validated instruments to measure handoff performance HOW ACCOMPLISH THIS? Specific Aims To create video-based examples of varying levels of handoff performance for education, adapting the approach in Arora et al MedEd Portal: “Handoffs: A Typical Day on the Wards” To validate an assessment instrument: the Handoff CEX 6 scripts developed & reviewed Changing handoff quality in the 3 domains of performance De-brief instrument development VIDEO DEVELOPMENT MedEd Portal Submission Arora V, Farnan J, Paro J, Vidyarthi A, Johnson J. Teaching Video:"Handoffs: A Typical Day on the Wards” MedEdPORTAL; 2011 Figure 1: Faculty Ratings of Communication 0 2 4 6 8 10 UnsatisfactorySuperior Setting p <0.001 Figure 2: Faculty Ratings of Setting 0 2 4 6 8 10 UnsatisfactorySatisfactory Superior Professionalism p <0.001 Figure 3: Faculty Ratings of Professionalism Ability to detect level of performance for communication impacted by level of professionalism depicted