Presentation on theme: "Systematic reviews of health promotion and public health interventions"— Presentation transcript:
1 Systematic reviews of health promotion and public health interventions Rebecca ArmstrongElizabeth WatersCochrane Health Promotion & Public Health Field
2 Overview Overview of systematic reviews Outline of The Cochrane CollaborationRole of the HPPH FieldFunction of systematic reviews in informing policy and practiceKey elements of systematic reviewsAsking answerable questionsSearching for evidenceAssessing qualitySynthesising resultsApplicability and transferability
3 Types of reviews Reviews Systematic reviews Meta-analysis (narrative/literature/traditional)Systematic reviewsMeta-analysis
4 Narrative reviews Usually written by experts in the field Use informal and subjective methods to collect and interpret informationUsually narrative summaries of the evidenceDo not confuse with narrative systematic reviews, which describes the type of synthesis of data.Read: Klassen et al. Guides for Reading and Interpreting Systematic Reviews. Arch Pediatr Adolesc Med 1998;152:
5 What is a systematic review? A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review**Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.
6 Key elements of a systematic review Structured, systematic process involving several steps :Formulate the questionPlan the reviewComprehensive searchUnbiased selection and abstraction processCritical appraisal of dataSynthesis of data (may include meta-analysis)Interpretation of resultsAll steps described explicitly in the review
7 Systematic vs. Narrative reviews Scientific approach to a review articleCriteria determined at outsetComprehensive search for relevant articlesExplicit methods of appraisal and synthesisMeta-analysis may be used to combine dataDepend on authors’ inclination (bias)Author gets to pick any criteriaSearch any databasesMethods not usually specifiedVote count or narrative summaryCan’t replicate review
8 Advantages of systematic reviews Reduce biasReplicabilityResolve controversy between conflicting studiesIdentify gaps in current researchProvide reliable basis for decision makingBut there can also be conflicting systematic reviews due to different methodologies used.
9 Limitations of systematic reviews specific to health promotion Results may still be inconclusiveThere may be no trials/evidenceThe trials may be of poor qualityThe intervention may be too complex to be tested by a trialPractice does not change just because you have the evidence of effect/effectiveness
10 Consider these interventions… Interventions to promote smoke alarm ownership and functionSchool-based driver education for the prevention of traffic crashesHelmets for preventing head and facial injuries in bicyclistsDo you think the results identified in SRs will be good, promising or absent (and potentially harmful)?
11 Results from systematic reviews Helmets reduce bicycle-related head and facial injuries for bicyclists of all ages involved in all types of crashes including those involving motor vehicles.The results provide no evidence that drive education reduces road crash involvement, and suggest that it may lead to a modest but potentially important increase in the proportion of teenagers involved in traffic crashes.Results from this review suggest that area-wide traffic calming in towns and cities may be a promising intervention for reducing the number of road traffic injuries and deaths. However, further rigorous evaluations of this intervention are needed.
12 The Cochrane Collaboration International non-profit organisation that prepares, maintains, and disseminates systematic up-to-date reviews of health care interventionsThe Cochrane Collaboration was established in October 1992.The Collaboration relies on funding from a number of different sources to complete its work.I like to think of a review as starting out as a jigsaw puzzle – especially if there is heterogeneity – ie. differences in interventions, settings, study designs, outcomes.
13 Cochrane Collaboration Named in honour of Archie Cochrane, a British researcherIn 1979:“It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials”
14 The Cochrane LibraryCochrane Systematic reviews : Cochrane reviews and protocolsDatabase of Reviews of Effects: Other systematic reviews appraised by the Centre for Reviews and Dissemination.Cochrane Central Register of Controlled Trials:Bibliography of controlled trials (some not indexed in MEDLINE).Health Technology Assessment Database: HTA reportsNHS Economic evaluation database:Economic evaluations of health care interventions.DATABASES:Cochrane Systematic reviews : Cochrane reviewsDatabase of Reviews of EffectivenessOther systematic reviews from around the world which have been appraised by the NHS Centre for Reviews and Dissemination.Cochrane Central Register of Controlled TrialsBibliography of controlled trials (some not indexed in MEDLINE).Cochrane database of Methodology ReviewsCochrane reviews of methodological studies.The Cochrane Methodology registerBibliography of studies relating to methodological aspects of research synthesisAbout the Cochrane Collaboration: Information about review groups, Fields, Centres, etc. Contact details provided.Health Technology Assessment Database:Health Technology Assessment reports from around the world.NHS Economic evaluation database:Economic evaluations of health care interventions.
17 Cochrane HPPH FieldRepresent the needs and interests of those in health promotion and public health in Cochrane mattersRepresent Cochrane in health promotion and public health forumsIn transition from Field to Review GroupWill edit PH reviews for the Cochrane Library
18 Cochrane Collaboration Structure Steering GroupReview GroupsCentresFieldsConsumer NetworkMethods Groups
19 Collaborative Review Groups (50) Produce systematic reviews relevant to a particular disease or health issue for inclusion in the Cochrane LibraryExamplesAirways GroupDrug and Alcohol GroupHeart GroupInjuries GroupSkin GroupPregnancy and Childbirth GroupStroke GroupBreast Cancer Group
20 Methods Groups (12)Provide advice and support in the development of the methods of systematic reviewsExamplesNon-Randomised StudiesScreening and Diagnostic TestsEmpirical Methodological StudiesQualitative Methods
21 Cochrane Centres (14)Work to assist all Cochrane entities within a specific geographical areaExamplesAustralasian Cochrane Centre (at Monash)South African Cochrane CentreItalian Cochrane CentreChinese Cochrane Centre
22 Cochrane Fields/Networks (9) Represent an area of interest which spans a number of health problems - and hence a number of Review GroupsExamplesHealth Promotion and Public Health FieldPrimary Health Care FieldCancer NetworkChild Health Field
23 Cochrane HPPH FieldCochrane Fields represent a population group, or type of care that overlaps multiple Review Group areasHPPH FieldRegistered in 1996Administered from MelbourneFunded by VicHealthOver 400 members on contact database across >30 countries
24 Staff Elizabeth Waters (Director) Jodie Doyle (Coordinator) Rebecca Armstrong (Senior Research Fellow)Naomi Priest (Research Fellow)
26 Questions of interest Effectiveness: Does the intervention work/not work?Who does it work/not work for?Other important questions:How does the intervention work?Is the intervention appropriate?Is the intervention feasible?Is the intervention and comparison relevant?
27 Answerable questions EFFECTIVENESS A description of the populations P An identified intervention IAn explicit comparison CRelevant outcomes O
28 A PICO question Time-consuming question: What is the best strategy to prevent smoking in young people?
29 An answerable question Q. Are mass media (or school-based or community-based) interventions effective in preventing smoking in young people?
30 The PICO(T) chart Problem, population Intervention Comparison Outcome Types of studiesYoung people under 25 years of agea) Televisionb) Radioc) Newspapersd) Bill boardse) Postersf) Leafletsg) Bookletsa) School-based interventionsb) No interventiona) objective measures of smoking (saliva thiocyanate levels, alveolar CO)b) self-reported smoking behaviourc) Intermediate measures (intentions, attitude, knowledge, skills)d) Media reacha) RCTb) Controlled before and after studiesc) Time series designs
33 Systematic review process Well formulated questionComprehensive data searchUnbiased selection and abstraction processCritical appraisal of dataSynthesis of dataInterpretation of results
34 A good search Clear research question Comprehensive search All domains, no language restriction, unpublished and published literature, up-to-dateDocument the search (replicability)Quality of evidence generated by a review depends entirely on the primary studies which make up the review.
35 Components of electronic searching Describe each PICO componentStart with primary conceptFind synonymsIdentify MeSH / descriptors / subject headingsAdd textwordsAdd other components of PICO question to narrow citations (may use study filter)Examine abstractsUse search strategy in other databases (may need adapting)
36 So you want to do a ‘quick & dirty’? DARECENTRALPubMed (clinical queries, related records)CDCNICEOrganisations who do work in your area…google
37 The Cochrane Library: www.thecochranelibrary.com
46 Searching www.google.com.au The order of terms will effect the results so start with the obvious or key conceptNo need for ‘and’Google will ignore common wordsIf they are important use + (e.g. policy + 3)Phrase searching is useful eg “suicide prevention”Google searches for variations on words eg diet, dietary
47 SearchingWhere terms have multiple meanings you can direct google to remove sites you want to avoid (e.g. bass –music)Keep your search strings briefMental health promotion initiatives to prevent suicide in young peopleCompartmentalise your search strings“mental health promotion” suicide“Suicide prevention” “young people”Prevention and suicide and youth
48 Searching http://www.ncbi.nlm.nih.gov/entrez/query.fcgi Use the same principles for google – keep it short and sweet.Key featuresJournals Database MeSH Database Single Citation Matcher Clinical Queries
53 These next few slides show you how to search MeSH terms in PubMed These next few slides show you how to search MeSH terms in PubMed. Useful if you don’t have access to electronic databases. You combine this method with the one for text words outlined above. The process for combining text words and MeSH terms is outlines below.
57 Find MeSH and textwords to ExampleMass media interventions to prevent smoking in young peopleP= Young peopleSTEP ONE:Find MeSH and textwords todescribe young people
58 Example Mass media interventions to prevent smoking in young people P= Young peopleMeSH: AdolescentChildMinors
59 Example Mass media interventions to prevent smoking in young people P= Young peopleTextwords:Adolescent GirlChild BoyJuvenile TeenagerYoung people Young adultStudent Youth
60 Textwords Truncation $: To pick up various forms of a word Teen$.tw Smok$.tw Teenage SmokeTeenager SmokingTeenagers SmokesTeens SmokerTeen Smokers
61 Textwords Wild cards ? and #: To pick up different spellings Colo?r.tw (? Can be substituted for one or no characters)ColourColorWom#n.tw (# Substitutes for one character)WomanWomen
62 Textwords Adjacent ADJn: retrieves two or more query terms within n words of each other, and in any orderGreat when you are not sure of phraseologyEg sport adj1 policySport policyPolicy for sportEg mental adj2 healthMental healthMental and physical health
63 Example continuedMass media interventions to prevent smoking in young peopleI = Mass media interventionsSTEP TWO:Find MeSH and textwords todescribe mass media interventions
64 Example continued MeSH Mass media Audiovisual aids Television Motion picturesRadioTelecommunicationsNewspapersVideotape recordingAdvertising
65 Example continuedMass media interventions to prevent smoking in young peopleO = Prevention of smokingSTEP THREE:Find MeSH and textwords todescribe prevention of smoking
66 Example of search OR OR OR P AND I C O O = PREVENTION OF SMOKING P = YOUNG PEOPLEMeSH Textwords………………………. ………………………. ………………………. ……………………….………………………. ……………………….I = MASS MEDIAC = (if required)O = PREVENTION OF SMOKINGPANDICOOROROR
67 Different bibliographic databases Databases use different types of controlled vocabularySame citations indexed differently on different databasesMedline and EMBASE use a different indexing system for study typePsycINFO and ERIC do not have specific terms to identify study typesNeed to develop search strategy for each database
68 Study design filters RCTs Non-RCTs Qualitative research See Cochrane Reviewer’s HandbookNon-RCTsNot yet developed, research in progressQualitative researchSpecific subject headings used in CINAHL, ‘qualitative research’ used in MedlineCINAHL Filter: Edward Miner LibrarySystematic reviews/meta-analysesCINAHL: as aboveMedlineMedline and EmbasePubMed
69 2. Unpublished literature Not all known published trials are identifiable in Medline (depending on topic)Only 25% of all medical journals in MedlineNon-English language articles are under-represented in Medline (and developing countries)Publication bias – tendency for investigators to submit manuscripts and of editors to accept them, based on strength and direction of results (Olsen 2001)
70 2. Unpublished literature Hand searching of key journals and conference proceedingsScanning bibliographies/reference lists of primary studies and reviewsContacting individuals/agencies/ academic institutionsNeglecting certain sources may result in reviews being biasedHandsearching: Not all journals are indexed in databasesMay be indexed incorrectlyKnowing when to stop: depends on the return you are getting, resources, time constraintsUse Reference system for collecting references i.e. Endnote, Procite, IdeaList, Reference Manager
73 Critical appraisalThe process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision.Alison Hill, Critical Appraisal Skills Programme, Institute of HealthSciences, Oxford
75 Why appraise validity?Not all published and unpublished literature is of satisfactory methodological rigourJust because it is in a journal does not mean it is sound!Onus is on you to assess validity!Quality may be used as an explanation for differences in study resultsGuide the interpretation of findings and aid in determining the strength of inferences
77 Allocation of concealment Selection biasRecruit participantsAllocation of concealmentInterventionAllocationControlConfoundingExposed to interventionNot exposed to interventionIntegrity of interventionIntention-to-treatWithdrawalsFollow-upFollow-upBlinding of outcome assessmentOutcomeOutcomeData collection methodsAnalysisAnalysisStatistical analysis
78 Critical appraisal tools RCTsThe Quality Assessment Tool for Quantitative Studies (http://www.city.hamilton.on.ca/PHCS/EPHPP/).Non-RCTsCochrane Effective Practice and Organisation of Care Group (http://www.epoc.uottawa.ca/).The Berkeley Systematic Reviews Group (http://www.medepi.net/meta/)
80 Qualitative research… explores the subjective world. It attempts to understand why people behave the way they do and what meaning experiences have for people.Qualitative studies of experienceProcess evaluationSubjective experience of an intervention, or of an illness, condition, etc.Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.
81 CASP appraisal checklist Clear aims of research (goals, why it is important, relevance)Appropriate methodology (what, how, why)Sampling strategyData collectionRelationship between researcher and participantsEthical issuesData analysisFindingsValue of research (context dependent)
82 Other qualitative checklist Quality frameworkGovernment Chief Social Researcher’s Office, UK19 question checklist for process evaluations (EPPI-Centre)
83 Appraisal of a systematic review 10 questionsClearly-focused questionThe right type of study includedIdentifying all relevant studiesAssessment of quality of studiesReasonable to combine studiesWhat were the resultsPreciseness of resultsApplication of results to local populationConsideration of all outcomesPolicy or practice change as a result of evidenceCASP
85 Factors influencing effectiveness Type III error (integrity of intervention)Theoretical framework of interventionContextFor whom did the intervention work, why, in what circumstances, at what costSee also the Public Health Schema by Rychetnik and Frommer, 2002 for formulating summary statements.
86 Difficulties addressing inequalities Studies rarely present information on differential effects of interventionsCannot locate studies addressing inequalitiesMay need original data from authorsLow power to detect subgroup differencesAlmost impossible to determine whether studies report data by SES from literature searches. Useful to have a broad strategy. Information will come from a variety of sectors including social, education and legal as well as health.Low power – many primary studies are small, and stratified analyses by SES will therefore only include a small number in each group. The power to detect differences will be low.
88 Assessing the applicability and transferability of interventions Applicability – whether the intervention process could be implemented in the local setting, no matter what the outcome is.Is it possible to run this intervention in this local setting?Eg. provision of condoms in area where they are not acceptable for religious reasonsWang et al 2005
89 Assessing the applicability and transferability of interventions Transferability – if the intervention were to be implemented in the local setting, would the effectiveness of the program be similar to the level detected in the study setting?E.g. if the interventionists lack experience and have few skills in delivering the intervention then its effectiveness in the local setting may be lower than that demonstrated in the study settingWang et al 2005
91 Contact details Rebecca Armstrong email@example.com If I can’t help you I might be able to point you in the right direction.If you are interested in training and support for conducting SRs or increasing uptake within your organisation, region, state please let me know