Presentation on theme: "Systematic reviews and Meta-analyses"— Presentation transcript:
1Systematic reviews and Meta-analyses Alison Brettle,Research Fellow (Information)Salford Centre for Nursing, Midwifery and Collaborative ResearchUniversity of Salford
2AimsTo discuss the role of the systematic reviews and meta-analyses and cover issues involved in their critical appraisal and interpretation
3Systematic ReviewA review of all the literature on a particular topic, which has been systematically identified, appraised and summarised giving a summary answer.
4What is a systematic review? An overview of primary research studies conducted according to explicit and reproducible methodologyA rigorous method of summarising research evidenceShows what we know and don’t know about a topic areaProvides evidence of effectiveness (or not) by summarising and appraising relevant evidence
5Systematic reviews aim To find all relevant research studies (published and unpublished)To assess each study on basis of defined criteriaSynthesise the findings in an unbiased wayPresent a balanced and impartial summary of the findings taking any flaws into consideration
6Advantages of systematic reviews Summarise evidence, keep people up to date without reading all published research literatureAllow large amounts of data to be assimilated (eg by busy clinicians, policy makers etc)A clearer picture by collating results of researchReduce bias – removes reviewers personal opinions, preferences and specialist knowledgeExplicit methods - allow the reader to assess how review has been compiledMore reliable conclusions because of methods used
7Systematic review models Medical/Health careCochrane Collaboration, NHS Centre for Reviews and DisseminationUsually includes “high quality” research evidence – RCTsOften includes meta-analysis (mathematical synthesis of results of 2+ studies that addressed same hypothesis in same way)Social care/Social SciencesSCIE, EPPI Centre, Campbell CollaborationOften include wider range of studies including qualitativeOften narrative synthesis of evidence
8Systematic review process Define/focus the questionDevelop a protocolSearch the literature (possibly 2 stages scoping and actual searches)Refine the inclusion/exclusion criteriaAssess the studies (data extraction tools, 2 independent reviewers)Combine the results of the studies to produce conclusion– can be a qualitative or quantitative (meta-analysis)Place findings in context – quality and heterogeniety of studies, applicability of findings
9Methodology for a systematic review of randomised controlled trials1 Greenhalgh, T, BMJ 1997;315:
10What type of study design? How effective is paracetamol at reducing pain?Does smoking increase the risk of oral cancer?
12Experimental studies Randomised controlled trial Non-randomised controlled clinical trialEvaluating the effectiveness of an intervention
13Observational studies Measuring the incidence of a disease; looking at the causes of disease; determining prognosisLooking at the causes of disease; identification of risk factors; suitable for examining rare diseasesMeasuring the prevalence of a disease; examining the associationCohortCase-controlCross-sectionalsurvey
14What is a meta-analysis? Optional part of a systematic reviewSystematic reviewsMeta-analyses
15Meta-analysisThe process of using statistical methods to combine the results of different studies.The aim is to integrate the findings, pool the data, and identify the overall trend of results(Dictionary of Epidemiology, 1995)
16Systematic ReviewsUnderstanding the jargon and the blobs!
17Odds Ratio, Relative Risk Measures of risk The likelihood of something happeningVThe likelihood of something not happeningMethods that describe/work out if something is causing harm (or having an effect)
18Odds Ratio Graph (Blobbogram) LEFTESMORIGHTELine of no significance2more than 10.5less than 11
19Odds Ratio 2 more than 1 0.5 less than 1 1 Best estimate Confidence Interval(wobble factor)2more than 10.5less than 11
20Odds Ratio (Blobbogram) 2more than 10.5less than 11Less than one is good – causing no harm (having an effect), middle line is no effect and right of line is causing harm/having no effect
21Confidence IntervalIs the range within which the true size of effect (never exactly known) lies, with a given degree of assurance (95% or 99%).
23Confidence Interval (CI) = the wobble factor, how sure are we about the results?- the shorter the CI the more certain we are about the results- if it crosses the line of 1 (no treatment effect) the intervention might not be doing any good and could be doing harm
24HeterogeneityClinical heterogeneity – differences in trial characteristicsStatistical heterogeneity - the variability in the reported effect sizes between studieshow similar are the results?are the differences among the results of the trials greater than could be expected by chance alone?
25Number needed to treat (NNT) the number of people you would need to treat with a specific intervention to see one additional occurrence of a specific outcomeAnother way of describing the benefits of treatment
26The p-value in a nutshell How often you would see a similar result by chance, whenactually there was no effect by the drug or treatment.Impossible Certain Absolutelyp= Very unlikely 1 in 1000p= Fairly unlikely 1 in 20p= Fairly likely 1 in 2p= Very likely 3 in 4The probability of something happening
27Critical appraisal Is the study valid? What are the results? TrustworthyWhat are the results?Is it useful in practice?Relevant?Generalisable?
28Evaluating quality of systematic reviews Is there a clearly defined question?Thorough and comprehensive searchWas methodological quality assessed and studies weighted accordingly? (Were studies reliable and valid?)How sensitive are the results to the way the review was done – ie if you changed the inclusion criteria how would this affect results?Interpretation of numerical resultsThings you are looking for to establish if it is valid
29Further readingGreenhalgh T (1997) How to read a paper: papers that summarize other papers (systematic reviews and meta-analyses), BMJ, 315:
30Useful resources Cochrane Collaboration http://www.cochrane.org/ Centre for Reviews and DisseminationFinding studies for systematic reviewsEPPI-Centre – Stages of a reviewSCIE - The conduct of systematic research reviews for SCIE knowledge reviews