1. Building and Measuring Safety Culture: Getting Started & Planning Your Survey Process January 10, 2012

2. Polling Question 1.Has your organization conducted a safety culture survey in the past 12 months? a.Yes b.No c.Do not know 2

3. Polling Question 2.If yes, what survey instrument was used? a.Safety Attitude Questionnaire (SAQ) b.Hospital Survey on Patient Safety (HSOPS) c.Other survey or measurement tool d.Do not know 3

4. Learning Objectives 1.Define safety culture 2.Describe why safety culture is important for improvement efforts 3.Identify best practices for measuring safety culture –Define the role and responsibilities of the survey coordinator –Identify characteristics of successful surveys and action items 4

5. A BRIEF OVERVIEW OF SAFETY CULTURE Part I 5

6. The Armstrong Institute Model to Improve Care Comprehensive Unit based Safety Program (CUSP) 1.Educate staff on science of safety 2.Identify defects 3.Assign executive to adopt unit 4.Learn from one defect per quarter 5.Implement teamwork tools Translating Evidence Into Practice (TRiP) 1.Summarize the evidence 2.Identify local barriers to implementation 3.Measure performance 4.Ensure all patients get the evidence Engage Educate Execute Evaluate Reducing Ventilator Associated Pneumonia HOB elevation SAT and SBT Oral Care with CHG CSS-ETT Structural measures http://www.hopkinsmedicine.org/armstrong_institute Technical WorkAdaptive Work Pre-Work: Measure clinician and staff perceptions of safety culture (HSOPS survey)

7. What is Safety Culture?  Perceived priority of safety relative to other goals  Culture is the compass team members use to guide their behaviors, attitudes, & perceptions on the job What will I get praised for? What will I get reprimanded for? What is the “right” thing to do? Culture provides the context for team success Image source: Marysia Tomaszewska, August 8, 2012, used under a Creative Commons LicenseMarysia Tomaszewska 7

8. What Are the Core Aspects of Safety Culture… Schein, 2007

9. Why Safety Culture Matters 1.Safety culture is related to outcomes –Patient outcomes  Patient care experience  Infection rates, sepsis  Postop. hemorrhage, respiratory failure, accidental puncture/laceration  Treatment errors –Clinician outcomes  Incident reporting, burnout, turnover 9 Huang et al., 2010; Mardon et al., 2010; MacDavitt et al., 2007; Singer et al., 2009; Sorra et al., 2012; Weaver, 2011.

10. Why Safety Culture Matters 2.Safety culture influences the effectiveness of other safety and quality interventions –Can enhance or inhibit effects of other interventions 3.Safety culture can change through intervention –Best evidence so far for culture interventions that use multiple components 10 Haynes et al., 2011; Morello et al., 2012; Van Nord et al., 2010; Weaver et al., in press

11. MEASURING SAFETY CULTURE Part II 11

12. CUSP & Safety Culture Safety Culture is typically measured “Pre-CUSP” (before interventions begin): –Provides a baseline to diagnose barriers and facilitators that can impact improvement efforts –Then can be measured 12-18 months following start of improvement efforts Use reliable and valid survey instrument –Hospital Survey on Patient Safety (HSOPS) CUSP is the intervention that you will use to help you improve culture results 12

13. What is the Hospital Survey on Patient Safety (HSOPS)?  Part of a suite of survey tools (SOPS) for hospitals, medical offices, nursing homes Sponsored by: Agency for Healthcare Research & Quality Developed by Westat, public release in 2004  The HSOPS App: An online survey tool  Developed by the Armstrong Institute, in partnership with CeCity  Allows participants to complete the survey online and survey coordinators to access detailed reports of results  Participants are asked to choose 1 to 5 for each question:  1 Strongly Disagree  2 Disagree  3 Neither Agree nor Disagree  4 Agree  5 Strongly Agree  1 Never  2 Rarely  3 Sometimes  4 Most of the time  5 Always 13

14. HSOPS Questions & Composite Scores 14 10 Composite Scores (“Dimensions”) No. of Questions Example Question 1. Supervisor/manager expectations & actions promoting patient safety 4 B1. My supervisor/manager seriously considers staff suggestions for improving patient safety. 2. Organizational learning-continuous improvement 3A9. Mistakes have led to positive changes here 3. Teamwork within unit4A1. People support one another in this unit. 4. Communication openness3 C4.Staff feel free to question the decisions or actions of those with more authority. 5. Feedback & communication about error3 C1.We are given feedback about changes put into place based on event reports. 6. Nonpunitive response to error3 A8.Staff feel like their mistakes are held against them. (negatively worded) 7.Staffing4A2.We have enough staff to handle the workload. 8.Hospital management support for patient safety 3 F8. The actions of hospital management show that patient safety is a top priority. 9.Teamwork across hospital units4 F4.There is good cooperation among hospital units that need to work together. 10.Hospital handoffs & transitions4 F5.Important patient care information is often lost during shift changes. (negatively worded)

15. HSOPS Questions & Composite Scores (cont’d) 15 6 Background questions 4 Outcome VariablesNo. of Questions Example Question 1. Overall perceptions of safety 4A15.Patient safety is never sacrificed to get more work done. 2. Frequency of event reporting 3 D1.When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported? 3. Patient safety grade (of hospital unit) 1E1.Please give your work area/unit in this hospital an overall grade on patient safety. 4. Number of events reported in the last 12 months 1G1.In the past 12 months, how many event reports have you filled out and submitted?

16. HSOPS Scoring Scoring guidelines created by AHRQ Scores represent the % of positive responses –% who gave a score of 4 or 5 16

17. OVERVIEW OF THE SURVEY ADMINISTRATION PROCESS Part III 17

18. Effective surveys… 1.Are confidential and anonymous No way to link responses with email address or other individual information Results reported at unit-level Critical that the survey coordinator, project team, and leadership focus on survey as a tool for learning and improvement 2.Have a clear reason and motivation to complete them Must be clear to staff why you are asking them to complete this survey, what will happen with results, and what will be done based on their input 3.Are publically supported by both formal and informal unit leaders If leaders and colleagues view the survey as important and worthy of their time, frontline staff will be more motivated to contribute their input 18

19. Two Options for HSOPS Option 1Option 2 Collect fresh HSOPS data from participating work areas Use online HSOPS App to survey work area team members Upload previously collected HSOPS data from participating work areas Upload excel file to online HSOPS App 19

20. What are the key tasks for planning our survey? PhaseTask Prep (~3 wks) wk 1 wk 2 wk 3 wk 4 wk 5 wk 6 wk 7 wk 8 wk 9 Planning stage Logistical tasks Who will administer? Who will be surveyed? Publicize! Motivate participation. Create a debriefing plan Alert participants Provide pre-notification (post, email, meetings) Upload unit dataLog-on and enter unit background info Start! Upload participant email addresses or previously collected data TrackMonitor response rates (target >60%) Remind1 st reminder Remind againFinal reminder End the surveyClose the survey Collect dataDownload results report ExecuteDebrief & plan improvements

21. Who are key personnel in the HSOPS process? 21

22. Survey Coordinators: Key Responsibilities 22 Survey coordinators… Help to coordinate the survey administration process, Learn how to use the online survey database (via training webinars and conference calls) Help survey participants with the survey process, Enter data about the work areas participating in the survey into the online survey database, Monitor the survey response rate using the online survey database, Work with hospital and work area leadership to distribute survey materials and information, Communicate with other HSOPS Coordinators in the project

23. Online HSOPS Application (“HSOPS App”) 23

24. Developing HSOPS Survey List 1.Determine whom to survey  Clinical staff only?  Non-clinical support staff (e.g., environmental services)? 2.Compile unit email list  E-mail address  Hospital area/unit  Staffing category (e.g., physician, nurse, technician, administrative support) Make sure your list is up to date, check for: –Staff on administrative or extended sick leave, –Staff who appear in more than one staffing category or hospital area/unit, –Staff who have moved to another hospital area/unit, –Staff who no longer work at the hospital, and –Other changes that may affect the accuracy of your list email addresses 24

25. Pre-Notification Materials to Inform Unit Clinicians and Staff of HSOPS 1.Why the hospital is conducting the survey and how staff responses will be used, 2.Which hospital staff were selected to be surveyed (e.g., all staff, nursing staff, all clinical staff, a random sample of staff, etc.), 3.How much time is needed to complete the survey, Confidentiality or anonymity assurances, 4.Suggested reply timeframe and how to return completed surveys, 5.Incentives for which units will be eligible, if they respond (Optional), and 6.Contact information for the main hospital point-of-contact. 25 Sorra & Nieva, 2004

26. Why should we aim for a response rate of 60% or higher? Response rate = Number of respondents who completed the online survey divided by the total number of unit members who were invited to complete the survey Safety culture reflects the shared perceptions among unit members Response rate of 60% is minimum necessary for a relatively representative sample Important for validity and reliability of results 26

27. Thinking ahead: Creating a Debriefing Plan for your Survey Results Debriefing is… –A semi-structured conversation among frontline clinicians and staff that is usually led by a designated facilitator Purpose… 1.Encourage open communication, transparency, and interactive discussion about the survey results across all levels 2.To engage clinicians and staff in generating and implementing their ideas about how to create an effective safety culture in their work area 27

28. Some points to cover in your debriefing plan 28

29. We are here to support & guide: HSOPS Calls 1.Getting Started & Planning the Process Intro to measurement, basics of survey administration 2.The online HSOPS process A more detailed, technical call for survey coordinators that will walk through the details of how to use the online survey app 3.Interpreting HSOPS data and debriefing Once data are collected and survey coordinators have download their results, another call will be held to walk through how to read reports, debriefing staff,and interpret results 29

30. What if we have other surveys going on?  It is common for hospitals to use several different surveys to elicit input from clinicians and staff  Efforts to measure safety culture can be aligned with other surveys E.g., JHM measures both safety culture and employee engagement annually 30 Some tips: Try to not have more than 1 survey going on at the same time if possible (Lack of time and survey fatigue can negatively impact results) Consider similar trends observed across surveys You can debrief results of similar surveys at the same time if similar trends or common themes are observed in results

31. Some Additional Tips for Effective Surveys Communicate to all staff on the unit the importance of their valuable input Check response rates periodically and send reminders at different times throughout the survey administration process: before the survey is administered and a few times during the survey administration process Send periodic reminders through various communication channels, such as emails, announcements, during staff meetings, etc. Create a debriefing plan for sharing results with clinicians and staff Don’t be afraid to reach out for assistance when in doubt 31

32. Action Items: 1.Identify a survey coordinator for participating work areas  …Or coordinators if your team desires more than one coordinator 2.Send collated list of survey coordinators to your coordinating entity (HEN, Hospital association, or affinity group leader)  If collecting new data or uploading previously collected data  Coordinating entity to compile and send to JHU 3.Determine whom to survey and sample size  …Or locate previously collected data file 4.If collecting fresh HSOPS data.:  Compile unit email list  Create informational materials to publicize the survey (posters, flyers, newsletters, meeting agenda items)  Send pre-notification letter, post pre-notification information, or hold pre-notification meeting 5.Create draft debriefing plan 32

33. Questions? Reminder… You can access all slides, call recordings, and project tools at the following website: https://armstrongresearch.ho pkinsmedicine.org/vap.aspx 33

34. References Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Dziekan G, Herbosa T, Kibatala PL, Lapitan MC, Merry AF, Reznick RK, Taylor B, Vats A, Gawande AA; Safe Surgery Saves Lives Study Group. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist- based surgical safety intervention. BMJ Qual Saf. 2011 Jan;20(1):102-7. Huang DT, Clermont G, Kong L, Weissfeld LA, Sexton JB, Rowan KM, Angus DC. Intensive care unit safety culture and outcomes: a US multicenter study. Int J Qual Health Care. 2010 Jun;22(3):151-61. MacDavitt K, Chou SS, Stone PW. Organizational climate and health care outcomes. Jt Comm J Qual Patient Saf. 2007 Nov;33(11 Suppl):45-56. Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf. 2010 Dec;6(4):226-32. Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Qual Saf. 2012 Jul 31. [Epub ahead of print] Schein E. Organizational culture and leadership, 4 th edition. San Francisco, CA: Jossey- Bass. 2010. 34

35. References Singer SJ, Falwell A, Gaba DM, Meterko M, Rosen A, Hartmann CW, Baker L. Identifying organizational cultures that promote patient safety. Health Care Manage Rev. 2009 Oct- Dec;34(4):300-11. Sorra J, Khanna K, Dyer N, Mardon R, Famolaro T. Exploring Relationships Between Patient Safety Culture and Patients' Assessments of Hospital Care. J Patient Saf. 2012 Jul 10. [Epub ahead of print]. Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. (Prepared by Westat, under Contract No. 290-96-0004). AHRQ Publication No. 04-0041. Rockville, MD: Agency for Healthcare Research and Quality. September 2004. van Noord I, de Bruijne MC, Twisk JW. The relationship between patient safety culture and the implementation of organizational patient safety defences at emergency departments.. Int J Qual Health Care. 2010 Jun;22(3):162-9. Weaver SJ. A configural approach to patient safety climate: The relationship between climate profile characteristics and patient safety. Doctoral dissertation. University of Central Florida. 2011. Weaver, S. J., Dy, S., Lubomski, L., & Wilson, R. Promoting a culture of safety. In R.M. Watcher, P.G. Shekelle, P. Pronovost (Eds.). Making healthcare safer: A critical analysis of the evidence of patient safety practices (AHRQ report # TBD). Rockville, MD. In press. 35