Presentation on theme: "The Bahrain Branch of the UK Cochrane Centre In Collaboration with Reyada Training & Management Consultancy, Dubai-UAE Cochrane Collaboration and Systematic."— Presentation transcript:
The Bahrain Branch of the UK Cochrane Centre In Collaboration with Reyada Training & Management Consultancy, Dubai-UAE Cochrane Collaboration and Systematic Review Workshop, February 2007, Dubai - UAE Dr. Zbys Fedorowicz, Dr. Dunia Al Hashimi, Dr. Ahmed Al Asfoor W02
Understanding Systematic Reviews
Reviews and Meta-Analyses Review: An article that summaries a number of different primary studies and may draw conclusions on the effectiveness of a particular intervention. A review may or may not be systematic. Meta - analysis by definition is a process of synthesizing research results by various statistical methods to retrieve, select, and combine results from previous separate but related studies. Meta - analyses are reviews that use systematic methods to summarizes the results of previous studies
Types of Reviews Narrative (Traditional) Systematic/Overview
A Systematic Review: A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critical appraise relevant primary research, and extract and analyze data from the studies that are included in the review. Statistical Methods (meta-analysis) may or may not be used.
Differs from traditional reviews in that they use a replicable, scientific and transparent approach to minimise bias.
FeatureNarrative ReviewSystematic Review QuestionsOften broad in ScopeOften a focused question Sources and SearchNot usually specified, potentially biased Comprehensive sources and explict search strategy SelectionNot usually specified, potentially biased Criterion-based selection, uniformly applied AppraisalVariableRigorous critical appraisal SynthesisOften a qualitative summary Quantitative summary * InferencesSometimes evidence- based Usually evidence-based * Quantitative summary that includes a statistical analysis is a meta-analysis
Systematic review Meta- analysis Literature review, systematic review, meta- analysis Literature review
Anatomy of a Systematic Review 1. The Question 2. Survey the review literature 3. Further define question 4. Design the Protocol 5. Search and Study Selection 6. Data Extraction 7. Analysis 8. Reporting
1.The Question Systematic reviews usually have a focused: Population/problem, setting(s). Intervention (treatment, diagnosis, technology, etc.) Outcome (patient, economic, usability, etc.) (EBM uses PICO as a guide to question formation)
2. Literature Review Check to see if a review has already been done on your question. This process can help re-define and focus your question. You can gain an understanding of the literature in the subject area.
3. Further Refine Your Question
4. Develop the Protocol The protocol is your research plan and should include: - The background The problem The methodology Protocols should be developed prospectively in limit bias.
Protocols should include: Specific questions to be answered Strategies and sources for finding information Inclusion/Exclusion criteria including Population, settings, problem Exposure, intervention Controls Outcome specified : definitions, what is being measured, how it’s being measured, where and by whom.
Cont.. – Type(s) of study design Types of studies to include –experimental, observational, qualitative, etc. Defined outcomes measuring what, how, and by whom Study selection Data Extraction strategy Analysis strategy Presentation of results
5. Finding the information Start with the question Decide on sources to search Develop search strategies Keep records and use a bibliographic software program to manage the information
Search Strategy Is an iterative process developed through input from the review team. Generates the lists of published (and unpublished) research studies Affects the validity of the results of the review The protocol guides the search by stating sources to search, types of information to include (published & unpublished), languages, study design, key concepts.
Search strategy cont’d The thoroughness of the searching is one of the defining differences between systematic and traditional reviews Must decide whether to go for a Sensitive search strategy – increases retrieval and irrelevant results OR Specific search strategy – increases precision but may miss relevant material
Sources of Information Databases of primary research articles Hand searching core subject journals Checking cited references Conference proceedings Contacting researchers and experts Dissertations Grey Literature Web of Science – citation tracking Internet
Why so thorough? Missing studies can have significant effect on the results of the review Publication Bias The tendency for studies with positive results to get published and studies showing no difference or negative results not getting published.
Thorough searching much research is never published not all research is published in journals not all research published in journals is indexed on major databases not all research indexed on databases can be easily retrieved
Sources of bias in trials Target population Sources of bias Allocation Selection bias Performance bias Attrition bias Detection bias Intervention Control group group Exposed to Not exposed to intervention Follow-up Outcomes
‘Bias’ publication bias Positive results language bias Positive results more likely to be published in English language publications geographical coverage bias of journals and databases
Why do we search widely? Savoie et al estimated that 29.2% of items in their review were uncovered by: searching the web handsearching scanning reference lists personal communication searching specialised databases and web sites.
Publication bias Song et al. Publication and related biases. Methods to dealing with publication and related biases in systematic reviews: literature searching locating unpublished studies assessment of the risk of publication and related biases detecting publication bias
Why search widely…. Allen & Hanburys found that: only 51% of the clinical trials relating to their respiratory products were published in journals indexed by MEDLINE, EMBASE or CINAHL (46% are in grey lit) Wallace et al. 11 of 65 trials in end stage renal disease reviews were found by searching beyond major databases.
Why search widely.... Long lead times before publication: publication gaps after conference presentation indexing publication lag before recorded in databases Cheng et al, % of a set of conference papers achieved publication within 12 months 29% within 2 years and 40% within 5 years
Delay and non publication Non-publication Cheng et al, 1998 only 32% of abstracts presented at chronic fatigue conferences were subsequently published in full Petticrew et al, % of oral paper presented at the Society for Social Medicine 1996 achieved publication
Key factors thorough searches searching beyond published articles citation searching Internet searching hand searching searching for ongoing research
Sources of Information REVIEWS: Cochrane Database of Systematic Reviews (CDSR) DARE Cochrane Central Database of Controlled Trials (CCDCT) other sources such as Health TechnologyAssessment sources HTA ( or CCOHTA – Canadian Coordinating Office for Health Technology Assessment (has a number of publications) International Network of Agencies for Health Technology Assessment
OTHER: Reference Lists & Citations Web of Science cited reference searches Hand searching key journals Researchers in the subject area Conference Proceedings (PapersFirst, Proceedings First) Dissertations (Dissertation Abstracts International) Internet
Grey Literature Databases of grey literature Grey Literature Report SIGLE System for Information on Grey Literature in Europe contains citations to reports and non- conventional (grey) literature issued informally throughout EC member countries. NetPrints -A repository of non-peer reviewed original research Organizations …. Clinical Trials
6. Information from included studies Bibliographic details Study characteristics - design/methods - participants - interventions - outcome measures Study results - means, SD and/or SE - no. of events and N
Make sense of a review A. Are the results of the review valid? B. What are the results? C. Will the results help locally?
A. Are the results of a review valid? 1.Did the review address a clearly focused issue? 2. Did the authors look for the appropriate sort of papers? 3. Do you think the important, relevant studies were included? 4. Did the review’s authors do enough to assess the quality of the included studies? 5. If the results of the review have been combined, was it reasonable to do so?
B. What are the results? 6. What is the overall result of the review? ‘Bottom line’ results; NNT, OR, RR, RD? 7. How precise are the results? Confidence interval, p values
C. Will the results help locally? 8.Can the results be applied to the local population? 9.Were all important outcomes considered? 10. Are the benefits worth the harms and costs?