1. Building on the Measurement of CFIR Constructs in FQHCs: Where Do We Go From Here? Maria Fernandez, PhD on behalf of the CPCRN FQHC WG Investigators CPCRN Steering Committee - Virtual Spring Webinar April 16, 2014

2. CPCRN CHC Survey FQHC Workgroup Partnership Committee Qualitative Inquiry Subgroup Data Subgroup CHC Survey Subgroup Align with CHCs’ missions Guided by real world health policy & health care delivery landscapes Health Care Reform Meaningful Use of EHR Patient-Centered Medical Home National Association of Community Health Centers (NACHC) Primary Care Associations Community Health Centers (CHCs)

3. Developing Measures of Constructs Associated with EBI Implementation Purposes Describe the survey development process and rationale Present initial psychometric validation of CFIR Inner Setting measures Discuss findings and what is next in our work assessing CFIR constructs (and PAR) related to EBI implementation Value One of the first studies to measure several constructs from CFIR Developed and tested measures that describe factors influencing implementation of EBAs in FQHCs Sets the foundation for establishing causal pathways and developing interventions that target these factors

4. Consolidated Framework for Implementation Research (CFIR) Source: Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science 2009; 4:50. Note: Authors from the VA and University of Michigan, SPH, Department of Health Management and Policy

5. CFIR Constructs Patient Needs & Resources, External Policies & Incentives Available Resources, Leadership Engagement, Culture, Implementation Climate, Learning Climate Knowledge & Beliefs about EBAs Executing, Reflecting & Evaluating Relative Advantage, Complexity Outer Setting Process Characteristics of Individuals Inner Setting Characteristics of Intervention

6. Multiple Recruitment Strategies Sites partnered with their state’s PCAs; PCAs emailed their CHCs Sites recruited CHCs via email, telephone calls, or in- person meetings Introductory email with online survey link; 4 reminder emails; in-person meeting (one site) January - May 2013 IRB approval at each site and coordinating center

7. Inner Setting Constructs Available Resources Culture Overall Culture Effort Culture Stress Implementation Climate Learning Climate Readiness for Change

8. Analysis plan We conducted separate CFAs for each of the seven inner setting constructs. Multiple fit indices were used to evaluate CFA model fit (CFI >.90, TLI >.90, SRMR <.08, and RMSEA <.08). Cronbach’s alpha to evaluate internal consistency reliability. Correlation coefficients of each pair of scales to examine discriminant validity. Inter-rater reliability (ICC(1) & ICC(2)) and inter-rater agreement ( r WG(J) ) statistics were computed to assess the reliability and validity of computing clinic- and systems-level means from the individual-level data.

9. Available Resources A35A A35B A35C C20A C20B C20C C20D.419.546.580.708.766.626.741 A11 A12 A13 A14 Readiness for change.824.896.847.872 C11 C12 C13 C05 Implementation climate.546.830.759.462 A01 A06 A02 A15 A19 Learning climate.694.646.768.727.735 A03 A05 A16Rev A07 A22 A08 A09 A10 A21 A36 A37 A38 A39 A40 A21 A42Rev A43 A44Rev Culture.663.723.556.642.742.674.719.731.775.332.438.387.594.480.612.440.581.387 Inner Setting 0.627 0.945 0.980 0.514 0.968

10. Combined CFA results and fit statistics CFA Standardized Factor Loadings a 1. Available Resources 0.627 (0.068) 2. Culture 0.945 (0.022) 3. Implementation Climate0.514 (0.068) 4. Learning Climate0.980 (0.024) 5. Readiness for Change0.968 (0.017) Robust Std. Errors in parenthesis CFI = 0.848 TLI = 0.835; SRMR = 0.079; RMSEA = 0.065

11. Discriminant validity of the five dimensions of inner setting ScaleCulture Available resources Learning climate Implementation climate Readiness for change Culture1.00 Available resources 0.36411.00 Learning climate0.67570.45551.00 Implementation climate 0.24640.62540.32011.00 Readiness for change 0.69780.52030.81140.37581.00 Correlations using average score for each scale *p <.05

12. Clinic Level Inter-rater reliability and inter-rater agreement statistics ScaleICC(1)ICC(2)r WG(J) Culture.182*.479.929 Available resources.116*.321.821 Learning climate.178*.476.785 Implementation climate.219*.504.828 Readiness for change.169*.462.634 Using average score for each scale *p <.05

13. System level Inter-rater reliability and inter-rater agreement statistics ScaleICC(1)ICC(2)r WG(J) Culture.181*.725.925 Available resources.114*.542.867 Learning climate.150*.679.774 Implementation climate.292*.792.845 Readiness for change.106*.558.592 Using average score for each scale *p <.05

14. Conclusions Our measures have structural validity Our measures have reliability Our measures have discriminant validity Our measures can be aggregated to the clinic level Clinics within systems have more similar values for inner-setting domain constructs than do clinics in different systems We fit a second order “inner setting” factor

15. PRACTICE ADAPTIVE RESERVE AND PATIENT CENTERED MEDICAL HOME BEST PRACTICES AT COMMUNITY HEALTH CENTER CLINICS IN SEVEN STATES

16. Adjusted Logistic Regression Frequency of PCMH Best Practices and PAR Scores PARFrequency of PCMH Best Practices (6-8 vs. 0-5) OR95% CI 0.08 – 1.005.492.31,13.06 0.60 - <0.802.231.11,4.47 0.00 - <0.60Referent PCMH CRC Screening Best Practices Dichotomized Score (6-8 vs. 0-5) Respondent reported performing PCMH best practices “usually” or “always” Adjusted for state, age, job type, years worked at the clinic, hours worked each week National Demonstration Project - Highly-motivated practices w/ significant capability for change Mean baseline PAR score 0.69 (s.d. 0.35) Post intervention PAR score increased to 0.74

17. Discussion What have we learned about some of the measurement challenges in FQHCs? Where do we go from here in terms of measurement? Which constructs need more conceptual and measurement work? What additional work is needed in terms of reliability and validity assessment?

18. Discussion Continued Where do we go from here in EBI implementation in FHQCs? What opportunities exist? How can the measurement work be used in EBI implementation in FQHCs?

19. How FQHC WG can move forward? How could this group move forward on what’s next? What resources, partners, and preliminary work is needed to pursue what’s next?