1. Narrative Synthesis in Systematic Reviews
An ESRC Research Methods Programme project
Mark Rodgers
Lisa Arai, Nicky Britten, Mark Petticrew, Jennie Popay, Helen Roberts, Amanda Sowden
Hello
Quick overview of our ESRC funded project
To devise guidance on narrative synthesis in systematic reviews.

2. Structure of the Presentation
What is “narrative synthesis”?
Where does narrative synthesis (NS) fit in a systematic review?
Why do we need guidance?
Developing a framework for NS
Building on the framework
Start with a quick overview

3. What Is a “Narrative Synthesis”?
NS are findings summarised and explained in words
As used here, NS refers to the approach adopted to bringing together the findings from studies included in a systematic review
No clear definition of NS
“Not a meta-analysis”
Can be alternative or complementary to MA
So what is a narrative synthesis?
Meta-analysis would be inappropriate where studies were too different from one another either statistically or clinically to combine using that method.
Nevertheless, we still want to be able to combine the findings of multiple studies in one place.

4. Where does it fit? ‘Typical’ systematic review process:
Define review question
Literature search
Study selection
Data extraction
Study appraisal
Synthesis
Conclusions/recommendations

5. Why Do We Need Guidance? NS lacks transparency
NS lacks reproducibility
Variations in practice
No coherent guidance currently exists
Why do we need guidance?
Unlike meta-analysis, lacks transparency and reproducibility.
Everybody does it a bit differently

6. Developing guidance for NS
Difficult (impossible?) to produce prescriptive guidance on the conduct of NS
Most appropriate approach depends on context
A NS “toolkit” may be better than traditional “guidance”
Requires a structure
Unfortunately, cannot produce singular, prescriptive guidance.
Part of the reason why so much variation exists in current approaches is that each synthesis has to be appropriate to the context in which it was conducted.

7. A framework for NS An descriptive framework of the NS process:
Developing a theory
Developing a preliminary synthesis
Exploring relationships in the data
Assessing the robustness of the synthesis

8. Developing a theory
“Theory of change” linking resources, activities, intermediate outcomes and ultimate goals
How the intervention works, why and for whom
Has implications for review question and inclusion criteria as well as interpretation
Consider early in the review process
Can be presented in narrative and/or diagrammatic form

9. Developing a preliminary synthesis
Initial description of the results of included studies
Organise results to describe patterns, e.g. in:
size/direction of effects
factors/processes impacting on implementation
quality
Product of this process is preliminary and should be interrogated further

10. Exploring relationships in the data
Explore relationships:
Between study characteristics and their findings
Between the findings of different studies
Explore influence of heterogeneity:
Variations in outcomes, methods, populations, interventions, settings etc
Theory may help develop plausible explanations for observed differences
Investigation of context partly dependent upon reporting of included studies

11. Assessing the robustness of the synthesis
Robustness of NS depends on two related elements:
Methodological quality of included studies
‘Trustworthiness’ of the synthesis product
Availability of key information in primary studies
Overall assessment of the strength of evidence available to support conclusions

12. Building on the framework (1)
Populated framework with range of “tools and techniques” (e.g. tabulation, idea webbing/conceptual mapping, transforming and translating data)
Allow reviewer to select tools/techniques most appropriate to the data being synthesised
Not necessarily linear or sequential process – reviewers will move iteratively between elements

13. Building on the framework (2)
Conducted two ‘test’ NS of domestic smoke alarm promotion interventions: one effects, one implementation
Qualified success - the approach can be further developed and refined
Would benefit from application to a wider range of syntheses