1. Systematic reviews and meta-analyses: when and how to do them Andrew Smith Royal Lancaster Infirmary 18 May 2015

2. Archie Cochrane 1909-88

4. Cochrane’s challenge ‘It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials’

5. Traditional reviews and the ‘desk drawer’ problem

6. Systematic review ‘A review in which evidence on a topic has been systematically identified, appraised and summarised according to predetermined criteria’

7. Systematic review 1. Decide scope and purpose (PICO) 2. Exhaustive search for material 3. Inclusion on pre-defined criteria 4. Quality assessment of relevant studies 5. Data extraction 6. Synthesis and integration 7. Interpretation

8. Primary and secondary research Reviews are retrospective, observational studies and are subject to both systematic and random error Bias can be reduced by applying the same principles in reviews as primary studies

9. Bias Primary Recruitment Selection Allocation Detection Attrition Reporting Secondary Selection Publication Language Inclusion Appraisal Interpretation

10. Funnel plots

11. Trial quality assessment

12. Meta-analysis

18. Heterogeneity: variability in estimates of treatment effects between studies in a meta-analysis Significant heterogeneity suggest that trials are not estimating a single common effect – possibly due to differences in patients, setting, intervention and outcomes Can be clinical, methodological or statistical Can be explored by subgroup and sensitivity analyses Meta-regression can also be applied Heterogeneity

19. Common ‘faults’ Too little thought Too many outcomes Too many subgroups ‘Lumping’ vs. ‘splitting’ Scoring risk of bias Not doing anything with quality assessment

20. Challenges in systematic reviews The RCT itself Anaesthesia/critical care Changing methodology New review types: DTA, non-randomised, prognostic, network meta-analysis False certainty

21. Trial sequential analysis

22. Challenges in systematic reviews The RCT itself Anaesthesia/critical care Changing methodology New review types: DTA, non-randomised, prognostic, network meta-analysis False certainty Uncertainty: inconclusive conclusions

23. The limits to knowledge doubtful unknown certain

24. When When you’re thinking about a research project When you need practical guidance When you really can’t do any primary research!

25. How Think first – write a protocol! Use PICO to specify the question Register the review Use ‘risk of bias’ tool for quality Get as many comments as possible Be explicit about process and judgements Be realistic about what can be achieved Use PRISMA tool for reporting