Narrative Synthesis in Systematic Reviews An ESRC Research Methods Programme project Mark Rodgers Lisa Arai, Nicky Britten, Mark Petticrew, Jennie Popay, Helen Roberts, Amanda Sowden Hello Quick overview of our ESRC funded project To devise guidance on narrative synthesis in systematic reviews.

Structure of the Presentation What is “narrative synthesis”? Where does narrative synthesis (NS) fit in a systematic review? Why do we need guidance? Developing a framework for NS Building on the framework Start with a quick overview

What Is a “Narrative Synthesis”? NS are findings summarised and explained in words As used here, NS refers to the approach adopted to bringing together the findings from studies included in a systematic review No clear definition of NS “Not a meta-analysis” Can be alternative or complementary to MA So what is a narrative synthesis? Meta-analysis would be inappropriate where studies were too different from one another either statistically or clinically to combine using that method. Nevertheless, we still want to be able to combine the findings of multiple studies in one place.

Where does it fit? ‘Typical’ systematic review process: Define review question Literature search Study selection Data extraction Study appraisal Synthesis Conclusions/recommendations

Why Do We Need Guidance? NS lacks transparency NS lacks reproducibility Variations in practice No coherent guidance currently exists Why do we need guidance? Unlike meta-analysis, lacks transparency and reproducibility. Everybody does it a bit differently

Developing guidance for NS Difficult (impossible?) to produce prescriptive guidance on the conduct of NS Most appropriate approach depends on context A NS “toolkit” may be better than traditional “guidance” Requires a structure Unfortunately, cannot produce singular, prescriptive guidance. Part of the reason why so much variation exists in current approaches is that each synthesis has to be appropriate to the context in which it was conducted.

A framework for NS An descriptive framework of the NS process: Developing a theory Developing a preliminary synthesis Exploring relationships in the data Assessing the robustness of the synthesis

Developing a theory “Theory of change” linking resources, activities, intermediate outcomes and ultimate goals How the intervention works, why and for whom Has implications for review question and inclusion criteria as well as interpretation Consider early in the review process Can be presented in narrative and/or diagrammatic form

Developing a preliminary synthesis Initial description of the results of included studies Organise results to describe patterns, e.g. in: size/direction of effects factors/processes impacting on implementation quality Product of this process is preliminary and should be interrogated further

Exploring relationships in the data Explore relationships: Between study characteristics and their findings Between the findings of different studies Explore influence of heterogeneity: Variations in outcomes, methods, populations, interventions, settings etc Theory may help develop plausible explanations for observed differences Investigation of context partly dependent upon reporting of included studies

Assessing the robustness of the synthesis Robustness of NS depends on two related elements: Methodological quality of included studies ‘Trustworthiness’ of the synthesis product Availability of key information in primary studies Overall assessment of the strength of evidence available to support conclusions

Building on the framework (1) Populated framework with range of “tools and techniques” (e.g. tabulation, idea webbing/conceptual mapping, transforming and translating data) Allow reviewer to select tools/techniques most appropriate to the data being synthesised Not necessarily linear or sequential process – reviewers will move iteratively between elements

Building on the framework (2) Conducted two ‘test’ NS of domestic smoke alarm promotion interventions: one effects, one implementation Qualified success - the approach can be further developed and refined Would benefit from application to a wider range of syntheses http://www.lancs.ac.uk/fass/projects/nssr/