1. Evidence Synthesis for Quality Improvement Purposes Davina Allen Nursing & Midwifery Studies

2. Direction of Travel Understanding the challenges of evidence synthesis for quality improvement: The case of integrated care pathways Breadth of evidence types Diversity of questions New research designs New models of evidence synthesis

3. Evidence for Quality Improvement Quality and patient safety are policy priorities Quality improvement initiatives are typically complex interventions Contextual factors shape effectiveness of intervention Need to know how interventions work in order to assess whether modification is required What works, for whom, in what circumstances and by what mechanism? Challenges of evidence generation, synthesis, translation

4. Integrated Care Pathways Introduced into healthcare in the US in 1980s Widespread use internationally Multidisciplinary care management tools Workflow system and record of care Bring evidence to the bedside Locally owned and maintained Pathway development and maintenance promotes inter-professional working

5. Active Ingredients? Development process (face to face MDT or e- working) ICP artefact (paper/EPR, record, guidelines, decision aides) ICP implementation/facilitation (education and training, coordinator) ICP maintenance (variance analysis, review and revisions)

6. Illustrative Cases Review 1Review 2 Allen, D. and Rixson, L. (2008) How has the impact of 'care pathway technologies' on service integration in stoke care been measured and what is the strength of the evidence to support their effectiveness in this respect? International Journal of Evidence-based Healthcare 6: 78- 110. Allen, D., Gillen, E., Rixson, L. (2009) A systematic review of the effectiveness of integrated care pathways: What works, for whom, in which circumstances? International Journal of Evidence- based Healthcare 7: 61-74

7. The Volume of Literature Large literature on ICPs ▫Diverse contexts ▫Breadth of patient populations ▫Wide range of purposes Review 1 ▫Search identified 10.000+ references ▫Narrowed to stroke: 2123 papers

8. Restricted Evidence for Inclusion Review 1 ▫2123 papers located ▫39 obtained for review ▫7 included in the study (5 studies) Review 2 ▫4055 papers located ▫35 obtained for review ▫9 papers included in the study (7 studies)

9. The Evidence Black Box Limited information on the intervention ▫Development, content, purpose, implementation Literature is a-theoretical Limited information on context Publication bias

10. New Directions Local Service EvaluationsAlternative Research Designs More information on: ▫ the intervention ▫ the context Identify theory to link intervention with outcomes Report successes and failures Reject efforts to establish whether pathways are unequivocally effective Realist evaluation (Pawson and Tilley, 1997) Not pass or fail verdict but examination of varying impacts across different contexts

11. New Directions Realist Synthesis (Pawson) Meta-Narrative Synthesis (Greenhalgh et al) Realist philosophy Unpack the relationships between context, mechanism and outcomes Suited to understanding ‘complex’ interventions where different contexts appear to produce different expected and unexpected effects Constructivist philosophy Illuminate the different paradigmatic approaches to a complex topic area Suited to topics where there is dissent about the nature of what is being studied and the best empirical approach to studying it

12. Connecting for Healthcare Improvement Descriptive study of nurses’ articulation work Realist synthesis of the literature on coordinating interventions In-depth ethnographic study of organisational/departmental interfaces and coordinating ‘interventions’/processes

13. Realist Synthesis of Coordinating Interventions (AKA swamp-dwelling) Meta Narrative SynthesisRealist Synthesis Science and technology studies Computer-supported cooperative work Medical Sociology Organizational Studies Boundary spanners Standards Pathways Routines Hand-offs Case management MDTs Handovers Ward rounds

14. Conclusion Healthcare managers need evidence-based information for quality improvement Quality improvement interventions are complex Healthcare systems create complex contexts Managers need to know: What works, in what context and how? Conventional research designs black box relevant information Conventional systematic review literature cannot cope with heterogeneity New research approaches needed ▫Realist synthesis New synthesis methodologies required ▫Meta-narrative analysis ▫Realist synthesis

15. Further reading Realist synthesis/review Wong G, Greenhalgh T, Westhorp G, Pawson D: Realist methods in medical education research: what are they and what can they contribute? Med Educ 2012, 46: 89-96. Pawson, R. (2006). Evidence-based Policy. A Realist Perspective. London: Sage. Meta-narrative synthesis Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R: Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med 2005, 61: 417-430. Join us at: www.jiscmail.ac.uk/RAMESES