1. A Longitudinal Study of an Intervention to Enhance Organizational Emphasis on Safety Academy Health June 9, 2008 Sara J. Singer Coauthors: Anita Tucker, Shoutzu Lin, Alyson Falwell, Jennifer Hayes Financial support was provided by AHRQ RO1 HSO13920 and Wharton’s Fishman Davidson Center

2. Motivation for studying Organizational Emphasis on Safety Organizational emphasis on safety (OES) can Organizational emphasis on safety (OES) can Improve safety performance (Weiner et al, 1997; Vogus & Sutcliffe, 2007; Zohar & Luria 2003) Improve safety performance (Weiner et al, 1997; Vogus & Sutcliffe, 2007; Zohar & Luria 2003) Shape worker behavior and attitudes (Carroll et al 2002, Edmondson 2003) Shape worker behavior and attitudes (Carroll et al 2002, Edmondson 2003) OES comes from managers who are visibly committed to safety (Gaba 2000, Weick 2001) OES comes from managers who are visibly committed to safety (Gaba 2000, Weick 2001) Less in known about mechanisms for increasing OES in organizations where it is low (Dixon and Shofer 2006) Less in known about mechanisms for increasing OES in organizations where it is low (Dixon and Shofer 2006) Notable exceptions Frankel et al 2005, Pronovost et al 2004 Notable exceptions Frankel et al 2005, Pronovost et al 2004

3. Possible mechanism for increasing Organizational Emphasis on Safety Low levels of OES may result from decoupling of Senior Managers (SM) from the frontlines Low levels of OES may result from decoupling of Senior Managers (SM) from the frontlines SM may be unaware of safety risks and opportunities for improvement SM may be unaware of safety risks and opportunities for improvement SMs shape resource allocation, process design, and organizational culture SMs shape resource allocation, process design, and organizational culture  Which influence frontline staff’s ability to provide safe patient care Challenge for SMs is leveraging frontline knowledge about system weaknesses Challenge for SMs is leveraging frontline knowledge about system weaknesses

4. Our intervention: Leveraging Front Line Expertise (LFLE) Over 18-month period, expose SMs to frontline work systems in context Over 18-month period, expose SMs to frontline work systems in context Frontline work system visit: Observe staff in context Frontline work system visit: Observe staff in context Safety communication forum: Listen to staff concerns about patient safety Safety communication forum: Listen to staff concerns about patient safety Debrief meeting: Prioritize “operational failures,” propose resolutions and assign responsibility for action Debrief meeting: Prioritize “operational failures,” propose resolutions and assign responsibility for action Feedback to staff: Communicate about actions taken Feedback to staff: Communicate about actions taken Through LFLE, SMs are more likely to Through LFLE, SMs are more likely to Recognize need for improvement Recognize need for improvement Allocate resources appropriately Allocate resources appropriately Make decisions supportive of safety Make decisions supportive of safety Repeat every 3 months on a different work area

5. One hospital’s implementation ED Med/SurgOutpatientPhysical Therapy Endoscopy

6. Hypotheses: Expected impact of LFLE on OES Hypothesis By increasing SM exposure to frontlines, which increases SM understanding of safety risks, Over time, Participation in the intervention…  SM perceptions (H1a)  nurse perceptions (H2a) Identification of more operational failures… SM perceptions (H1b)  SM perceptions (H1b) nurse perceptions (H2b)  nurse perceptions (H2b) More action to address failures…  SM perceptions (H1c)  nurse perceptions (H2c) More feedback to nurses… NA  nurse perceptions (H2d)

7. Methods 24 hospitals randomly selected (from a random, stratified sample of 92) for an 18-month intervention (2005 – 2006) 24 hospitals randomly selected (from a random, stratified sample of 92) for an 18-month intervention (2005 – 2006) 20 hospitals completed intervention 20 hospitals completed intervention Intervention hospitals did not differ significantly from US hospitals with respect to urban location and teaching status Intervention hospitals did not differ significantly from US hospitals with respect to urban location and teaching status Work areas where intervention was most commonly implemented: ED, ICU, OR/PACU, and Medical/Surgical Wards Work areas where intervention was most commonly implemented: ED, ICU, OR/PACU, and Medical/Surgical Wards

8. Data Pre (2004-5) and Post (2006-7) survey of personnel in 49 control hospitals and 20 intervention hospitals Pre (2004-5) and Post (2006-7) survey of personnel in 49 control hospitals and 20 intervention hospitals Sample: 100% senior managers (dept head and above); 10% nursing staff Sample: 100% senior managers (dept head and above); 10% nursing staff Response rates: 76% SM and 67% nurses in 2004, 62% SM and 50% nurses in 2006 Response rates: 76% SM and 67% nurses in 2004, 62% SM and 50% nurses in 2006

9. Measure Organizational emphasis on safety (OES) (  Organizational emphasis on safety (OES) (  Senior management considers patient safety when program changes are discussed. Senior management considers patient safety when program changes are discussed. Senior management has a clear picture of the risk associated with patient care. Senior management has a clear picture of the risk associated with patient care. I have enough time to complete patient care tasks safely. I have enough time to complete patient care tasks safely. My unit provides training on teamwork in order to improve patient care performance and safety. My unit provides training on teamwork in order to improve patient care performance and safety. Overall, the level of patient safety at this facility is improving. Overall, the level of patient safety at this facility is improving. All items used 5-point Likert response scale All items used 5-point Likert response scale Computed mean of all items, weighted equally Computed mean of all items, weighted equally

10. Analysis Senior managers Used two-level HLM to compare post-intervention perception of OES among SM in intervention hospitals (n=20) to non-intervention hospitals (n=49) Used two-level HLM to compare post-intervention perception of OES among SM in intervention hospitals (n=20) to non-intervention hospitals (n=49) Nurses Used three-level HLM to compare post-intervention perception of OES among nurses in intervention work areas (ED, ICU, OR/PACU, Med/Surg) (n=66) to same areas in non-intervention hospitals (n=172) Examined impact of intervention assignment and of greater implementation (# failures identified, % failures acted upon, and % failures with feedback) Examined impact of intervention assignment and of greater implementation (# failures identified, % failures acted upon, and % failures with feedback) Controlled for DV baseline and for individual, work area, and hospital characteristics Controlled for DV baseline and for individual, work area, and hospital characteristics

11. Impact of LFLE on Organizational Emphasis on Safety: Mean post-differences controlling for pre-differences CoefficientOES (SM)OES (nurses) 1. Intervention assignment.087 (0.21) -.166 (0.31) 2. Failure identification -.001^ (0.00)-.002 (0.00) 3. Percent failures acted upon.206* (0.10).018 (0.14) 4. Percent failures with feedback.116 (0.15).393* (0.19) Pre-intervention baseline in DV.865** (0.14).250** (0.06) Constant 1.889* (0.81) 2.976** (0.25) Observations 21131709 Number of groups 69238 A positive coefficient is associated with an improved perception (OES “got better”) ** p<0.01, * p<0.05, ^ p<0.1 Standard errors in parentheses SM model compared intervention v non-intervention hospitals; Nurse model compares work areas. Models included as control variables: age, tenure, discipline, teaching status, urban location, number of hospital beds, and in the nurse model only, work area.

12. Conclusions Our research represents one of the first quasi- experimental controlled studies to test an intervention to increase exposure of senior managers to frontlines Our research represents one of the first quasi- experimental controlled studies to test an intervention to increase exposure of senior managers to frontlines  Demonstrated ability to conduct intervention with senior managers in a healthcare setting Exposing senior managers to frontline safety hazards in context improved SM and nurses’ assessment of organizational emphasis on safety Exposing senior managers to frontline safety hazards in context improved SM and nurses’ assessment of organizational emphasis on safety  Which we expect in turn may improve patient safety

13. Thank You

14. Summary of results H1: SM perception of OES H1: SM perception of OES  Were not influenced by hospital participation in LFLE  Declined marginally with identification of more operational failures  Improved with more action to address failures 0.206 increase in OES represents a 5% change in the mean (.206/4.00) or 31% of a SD (0.206/0.66) 0.206 increase in OES represents a 5% change in the mean (.206/4.00) or 31% of a SD (0.206/0.66) H2: Nurse perception of OES H2: Nurse perception of OES  identification, or action  Were not influenced by work area participation, identification, or action  Improved with more feedback 0.393 increase in OES represents a 11% change in the mean (.393/3.50) or 49% of a SD (0.393/0.81) 0.393 increase in OES represents a 11% change in the mean (.393/3.50) or 49% of a SD (0.393/0.81) With more with action and feedback, the intervention produced an effect among senior managers in intervention hospitals and among nurses in intervention work areas With more with action and feedback, the intervention produced an effect among senior managers in intervention hospitals and among nurses in intervention work areas