1. Necessary but Not Sufficient: Why Strategic Implementation Climate and Molar Organizational Climate Both Matter for EBP Implementation
Nate Williams, PhD, LCSW
Assistant Professor,
Boise Statue University
School of Social Work

2. Q: How can mental health service organizations support EBP implementation? Q: How do molar climate and implementation climate relate to EBP implementation?
Q--What are the causal determinants, or levers, that they can pull to support EBP implementation?

3. Molar Organizational Climate & EBP Implementation Climate
Molar Climate
EBP Implementation Climate
Employees’ shared perceptions of the impact of the work environment on their personal well-being
Long research history in the organizational sciences
Common to all organizational environments
Mixed evidence linking it to implementation
Clinicians’ shared perceptions of their organization’s policies, procedures, and practices and the extent to which the organization expects, supports, and rewards the implementation of EBP
Considered most proximal to EBP implementation
Linked to implementation outcomes in manufacturing organizations, but…
Much less research and equivocal support in health services

4. Theoretical Models Linking Climate to EBP Implementation
Implementation Climate
Molar Climate
EBP implementation
Implementation Climate
Molar Climate
EBP implementation
Implementation Climate
Molar Climate
EBP implementation

5. An interactive Model of Climate Effects
We propose that EBP implementation climate and molar climate interact in their cross-level effects on clinicians’ EBP use
Sub-optimal EBP Implementation
Optimal EBP implementation
Poor EBP implementation
Sub-optimal EBP implementation
Low High
Implementation Climate
Positive
Molar
Climate
Negative
The equivocal results of prior research and theory on organizational climate and organizational support suggest an interaction may be present

6. Short- and Long-Term Effects of Climate
We propose that implementation climate and molar climate will have long-term (interactive) effects on clinicians’ EBP use
Climate
Climate
Climate

7. Study Hypotheses 1 & 2: Concurrent Effects of Climate
Time 1
Time 2
Implementation Climate
(T1)
Clinician EBP implementation H1
Molar Climate
(T1) H2

8. Study Hypotheses 3 & 4: Long-Term Effects of Climate
Time 1
Time 2
Molar Climate
(T1) H4
Implementation Climate
(T1)
Clinician EBP implementation
(T2) H3

9. Setting – City of Philadelphia

10. Timing of Data Collection
Pre-EPIC
19 agencies
23 sites
130 therapists
22 agencies
28 sites
247 therapists
21 agencies
26 sites
249 therapists

11. Study Sample (k = 20 sites)
Time 1
Time 2
Climate
n = 112 clinicians
response rate = 46%
n = 164 clinicians
response rate = 65%
M within-organization response rate = 59% (min = 23%, max = 92%)
M clinicians per organization = (min = 3, max = 39)
Total N = 235 n = 41 clinicians present at both times (17.4% of total sample)

12. Measures: Molar Climate
OSC Functionality Scale (Glisson et al., 2008)
15 items (α = .92)
Cooperation
There is a feeling of cooperation among my coworkers
Growth & Advancement
This agency provides numerous opportunities to advance if you work for it
Role Clarity
My job responsibilities are clearly defined

13. Measures: Implementation Climate
Implementation Climate Scale (Ehrhart et al., 2014)
18 items (α = .91)
Focus
Using EBP is a top priority in this agency
Recognition
Clinician who use EBP are held in high esteem in this agency
Educational Support
This agency provides EBP trainings or in-services
Rewards
The better you are at using EBP, the more likely you are to get a bonus or raise
Selection (EBP)
This agency selects staff who have previously used EBP
Selection (Openness)
This agency selects staff who are adaptable

14. Construct Validity of Organizational Climate Scores
Within-organization agreement
rwg(j) all > .70
range = .79 to .98
Between-organization variance
ICS - η2 = .41
OSC - η2 = .35

15. Outcome Measure: Clinician EBP Use
CBT subscale of the Therapy Procedures Checklist-Family Revised (Weersing, Weisz, & Donenberg, 2002)
Consider a specific, representative client you’re currently treating and endorse specific techniques from a list
33 items
α = .94 (T1), α = .92 (T2)
1-5 point range
33 items
Example items:
Teach child that cognitions affect mood and can cause emotional problems
Give direct instruction and information to change thoughts
Encourage appropriate self-talk
Teach problem-solving and decision-making strategies
Use time-out
Develop secondary reinforcers
Parent training in child management
Make up a contract

16. Preliminary Analyses – Variation in DV
Time 1
M = 3.25, SD = .70
ICC(1) = .29***
Time 2
M = 3.33, SD = .65
ICC(1) = .12**

17. Results: Concurrent Effects of Climate
Implementation Climate
(T1)
Clinician EBP implementation H1
Molar Climate
(T1) H2 * *
β= .38
Controlling for the other variables, every one standard deviation increase in implementation climate predicted a .38 standard deviation increase in clinicians’ expected use of EBP
H1 - β= .38, p = .043, R2Δ = .29
H2 - p = .016, R2Δ = .30
Control variables
- org size (N therapists)
- Clinician tenure
- Clinician hours per week
Climate
Time 1
Time 2

18. Concurrent Interaction between Molar and Implementation Climates
The standardized beta for positive molar climates is B=.35

19. Results: Long-Term Effects of Climate
Molar Climate
(T1) H4
Implementation Climate
(T1)
Clinician EBP implementation
(T2) H3 * *
β = .27
H3 was supported
β = .27, p = .028
R2Δ = 55%
H4 was supported
B = .01, p = .042
R2Δ = 1%
Higher levels of implementation climate at Time 1 predicted increased use of EBP among clinicians who were present in the organizations two years later (B = .34, SE = .14, p = .028), accounting for 55% of the organizational intercept variance beyond that accounted for by the other variables. For every one standard deviation increase in implementation climate at Time 1, clinicians’ expected use of EBP in these organizations at Time 2 increased by .27 standard deviations
Climate
Time 1
Time 2

20. Long-Term Interaction Between Molar and Implementation Climates
Standardized beta in positive molar climates = .23

21. What Does it Mean for Theory & Practice?
Confirmation of the role of implementation climate & identification of a boundary condition
Support for long-term stability of climate effects
Implications
Pre-implementation assessments of climate may be valuable for optimizing implementation efforts
Leaders must attend to the effects of the work environment on clinicians in addition to EBP issues

22. Study Strengths and Limitations
Correlational not experimental
Common method variance
Clinician reports of EBP use via standardized measure
CBT is not the only EBP for youth psychiatric disorders

23. Bottom Line for Executives
Organizations can optimize the delivery of effective treatments by developing organizational climates that engender high expectations and support for EBP implementation as well as strong support for clinician well-being

24. THANK YOU! Questions? Nate Williams, PhD, LCSW