1. Aligning Measurement-Based QI with Evidence-Based Practice Implementation Richard Hermann, MD, MS Associate Professor of Medicine and Psychiatry Tufts University School of Medicine Center for Quality Assessment & Improvement in Mental Health at Tufts-New England Medical Center www.cqaimh.org

2. 2 Overview How does evidence-based practice implementation (EBPI) relate to measurement-based quality improvement (MBQI)? – different paradigms – similarities and differences Potential for convergence & synergy What obstacles need to be addressed? Current research study on QI

3. 3 Evidence-Based Practices EBP Rating ACT / ICMA Evidence-based PsychotherapiesA Family PsychoeducationA Supported EmploymentA Integrated Dual Diagnosis Treatment A Medication Management A Multi-Systemic TherapyA A = RCTsB = less rigorous studiesC = consensus or opinion

4. 4 EBP Implementation: a Top-Down Model Research : Controlled trial of clinical intervention ↓ Development : Codification of EBP by experts ↓ Commercialization : Packaging: tools, scales, materials ↓ Diffusion : Social marketing, training, support ↓ Adoption : Local provider organizations ↓ Consequences : Change to practice & outcomes Rogers, Diffusion of Innovations, 2003

5. 5 Measurement-Based QI Measurement-Based QI A “bottom-up” model – Activities conducted by local provider organizations – Influenced by external groups MBQI is in wide use: – 90-98% of hospitals report formal programs MBQI is costly: – estimated cost ~$200,000 per hospital per year

6. 6 Principles of Measurement-Based QI Principles of Measurement-Based QI Quality as problems in “processes” Measurement & analysis Broad participation Inductive reasoning Trial and error

7. Model for Measurement-based QI

8. 8 Commonalities between MBQI and EBPI Both address important problems—some overlap Both employ measurement – MBQI: rates of EBP use, appropriateness – EBPI: fidelity to evidence-based model Both start with an understanding of underlying processes – MBQI: determined locally, informed externally – EBPI: studied externally, expanded locally Both involve systematic intervention to change practice – MBQI: determined locally, informed by research & experience – EBPI: developed by experts, customized to local circumstances

9. 9 Potential for MBQI to Enhance Evidence-Based Practice Implementation Promotes local organizational development – system perspective – team work – analytic skills – experience implementing change Increases awareness of gaps Prompts investigation Motivates exploration of available interventions → Potential for uptake of EBPs

10. Integrating MBQI with EBPI Requires Alignment Across Healthcare System

11. 11 Conditions for Successful Alignment 1. Local organizations need to select QI objectives that address gaps between actual & evidence-based practice 2. External organizations mandating measures also need to emphasize measures of EBPs 3. Microsystems within local organizations need to execute these QI activities effectively

12. 12 1. Do Quality Measures Used for Local MBQI Address Evidence-Based Practices? Reviewed measures developed for mental health QI 308 measures identified & evaluated: – 9% supported by RCTs – 30% supported by less rigorous evidence – 61% not supported by evidence Evidence-based measures less likely to be adopted Pilot study of QI objectives adopted by MA hospitals: < 10% of hospital objectives address EBPs National Inventory of Mental Health Quality Measures (www.cqaimh.org)

13. 13 2. Do Mandated Quality Measures Address Evidence-based Processes of Care? Measures established by: – Accreditor requirements – Government reporting requirements – Benchmarking collaboratives Results increasingly linked to: – Pay for performance incentives – Public disclosure – Employer purchasing decisions

14. 14 2. Do Mandated Quality Measures Address Evidence-based Processes of Care? Illustrative Measures Rating Restraint / seclusion ratesC Elopement rateC Injury rateC Number of medicationsC Readmission rateC Medication errorsB Antipsychotic doseA Antidepressant AdherenceA A = RCTsB = less rigorous studiesC = consensus or opinion

15. 15 Evidence-Based Practices EBP Rating ACT / ICMA Evidence-based PsychotherapiesA Family PsychoeducationA Supported EmploymentA Integrated Dual Diagnosis Treatment A Medication Management A Multi-Systemic TherapyA A = RCTsB = less rigorous studiesC = consensus or opinion

16. 16 Attributes Informing Quality-Measure Selection Represent Mental Health System Broadly Maximize Measure Attributes

17. 17 Evidence-Based Objectives for Inpatient QI: Schizophrenia ↑ use of antipsychotic drugs w/in recommended dose range ↓ use of multiple antipsychotics without adequate rationale ↑ % receiving adequate drug trials for refractory sx ↑ assessment/detection for EPS, akathisia or TD; ↑ rate of evidence-based treatment ↑ enrolled/referred to ACT among inpatients at high risk for relapse ↑ family members provided/referred to psychoeducation ↑ fidelity of inpatient psychoeducation program.

18. 18 Evidence-Based Objectives for Inpatient QI: Depression ↑ use of antidepressant drugs w/in recommended dosage range ↑ assessment/detection of psychosis among depressed inpatients; ↑ use of adequate pharmacotherapy or ECT for psychotic depression ↓ use of anticholinergic antidepressants among depressed elderly inpatients ↑ % of inpatients w/ major depression referred to OP clinicians providing evidence-based psychotherapy

19. 19 Other Evidence-Based Objectives for Inpatient QI ↑ assessment & detection of medical conditions ↑ % receiving appropriate inpatient medical care, outpatient referral & communication between IP & OP clinicians ↑ assessment/detection of SUD; ↑ % receiving inpatient treatment & OP referral

20. 20 3. Do Local Healthcare Organizations Execute QI Activities Effectively? Effectiveness in controlled trials Shortell (1998) reviewed 55 studies finding “pockets of improvement” rather than evidence of widespread change Effectiveness of routine QI Not well studied Case reports of successful initiatives Anecdotal evidence suggests much of local QI is ineffective

21. 21 Macro Model of Organizational Predictors of QI Shortell, 1995

22. 22 Predictors of QI Implementation Cultural: beliefs, values & behaviors relative to QI + organizational culture emphasizing teamwork & innovation + commitment of senior managers & physicians Structural: individual & group responsibilities + Decentralized decision-making + Longer experience + Greater number of teams & projects Strategic: approach to QI + “prospector” approach Technical: resources + presence of organization-wide information systems

23. 23 NIMH-funded study of 32 hospitals in MA & CA What are inpatient psychiatry units trying to improve? – effectiveness-- patient-centered care – access-- safety – equity-- efficiency To what extent do these objectives address EBPs? – Facilitators & barriers to adoption To what extent do hospitals achieve measurable change? Hypothesis – Fit between organization & predict QI effectiveness Study of MBQI in Inpatient Psychiatric Units

24. Micro Model of Organizational Predictors of QI Hermann, 2005

25. 25 Culture Inpatient clinicians’ knowledge & beliefs about evidence basis for QI objective Inpatient clinicians’ beliefs about the value of the QI objective to their patients’ care & outcomes

26. 26 Structure Course of QI objective as tracer of organizational structure: – serial reports of results disseminated to inpatient clinicians? – are interventions attempted? – reports of progress (or barriers) to appropriate committees? – participation / coordination among necessary departments?

27. 27 Leadership Selecting objectives that are priority of hospital leaders? Responsive to external pressures? Leaders actively involved or monitoring progress?

28. 28 Resources Availability of resources for achieving QI objective – training – tools – time – support (eg, data collection & analysis)

29. 29 Conclusion Greater progress toward implementing EBPs may be achieved by aligning organizations’ QI activities with EBP goals Components of alignment: – Provider organizations need to select evidence-based QI objectives – External groups need to reinforce emphasis on EBPs – Local MBQI needs to be more effective Ongoing research aimed at: – understanding barriers to adopting evidence-based QI objectives – understanding organizational factors influencing QI progress – developing interventions to improve effectiveness of local QI