Presentation is loading. Please wait.

Presentation is loading. Please wait.

The Bahrain Branch of the UK Cochrane Centre In Collaboration with

Similar presentations

Presentation on theme: "The Bahrain Branch of the UK Cochrane Centre In Collaboration with"— Presentation transcript:

1 The Bahrain Branch of the UK Cochrane Centre In Collaboration with Reyada Training & Management Consultancy, Dubai-UAE Cochrane Collaboration and Systematic Review Workshop, February 2007, Dubai - UAE W02 Dr. Zbys Fedorowicz, Dr. Dunia Al Hashimi, Dr. Ahmed Al Asfoor

2 Understanding Systematic Reviews

3 Reviews and Meta-Analyses
Review: An article that summaries a number of different primary studies and may draw conclusions on the effectiveness of a particular intervention. A review may or may not be systematic. Meta - analysis by definition is a process of synthesizing research results by various statistical methods to retrieve, select, and combine results from previous separate but related studies. Meta - analyses are reviews that use systematic methods to summarizes the results of previous studies

4 Types of Reviews Narrative (Traditional) Systematic/Overview

5 A Systematic Review: A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critical appraise relevant primary research, and extract and analyze data from the studies that are included in the review. Statistical Methods (meta-analysis) may or may not be used.

6 Differs from traditional reviews in that they use a replicable, scientific and transparent approach to minimise bias.

7 Feature Narrative Review Systematic Review
Questions Often broad in Scope Often a focused question Sources and Search Not usually specified, potentially biased Comprehensive sources and explict search strategy Selection Criterion-based selection, uniformly applied Appraisal Variable Rigorous critical appraisal Synthesis Often a qualitative summary Quantitative summary* Inferences Sometimes evidence-based Usually evidence-based * Quantitative summary that includes a statistical analysis is a meta-analysis

8 Literature review, systematic review, meta-analysis
Meta-analyses can be narrowly defined as quantitative pooling of results of individual studies. It is also sometimes called as overviews or systematic reviews. We know, in a systematic review, all relevant research are systematically searched, assessed and synthesized. Therefore, a narrowly defined meta-analysis is only part of a systematic review. This clarification is important since many systematic reviews are not able to quantitatively combined study results, and some meta-analyses are not systematic.

9 Anatomy of a Systematic Review
The Question Survey the review literature Further define question Design the Protocol Search and Study Selection Data Extraction Analysis Reporting

10 The Question Systematic reviews usually have a focused:
Population/problem, setting(s). Intervention (treatment, diagnosis, technology, etc.) Outcome (patient, economic, usability, etc.) (EBM uses PICO as a guide to question formation)

11 2. Literature Review Check to see if a review has already been done on your question. This process can help re-define and focus your question. You can gain an understanding of the literature in the subject area.

12 3. Further Refine Your Question

13 4. Develop the Protocol The protocol is your research plan and should include: - The background The problem The methodology Protocols should be developed prospectively in limit bias.

14 Protocols should include:
Specific questions to be answered Strategies and sources for finding information Inclusion/Exclusion criteria including Population, settings, problem Exposure, intervention Controls Outcome specified : definitions, what is being measured, how it’s being measured, where and by whom.

15 Cont.. – Type(s) of study design Study selection
Types of studies to include –experimental, observational, qualitative, etc. Defined outcomes measuring what, how, and by whom Study selection Data Extraction strategy Analysis strategy Presentation of results

16 5. Finding the information
Start with the question Decide on sources to search Develop search strategies Keep records and use a bibliographic software program to manage the information

17 Search Strategy Is an iterative process developed through input from the review team. Generates the lists of published (and unpublished) research studies Affects the validity of the results of the review The protocol guides the search by stating sources to search, types of information to include (published & unpublished), languages, study design, key concepts.

18 Search strategy cont’d
The thoroughness of the searching is one of the defining differences between systematic and traditional reviews Must decide whether to go for a Sensitive search strategy – increases retrieval and irrelevant results OR Specific search strategy – increases precision but may miss relevant material

19 Sources of Information
Databases of primary research articles Hand searching core subject journals Checking cited references Conference proceedings Contacting researchers and experts Dissertations Grey Literature Web of Science – citation tracking Internet

20 Systematic Searching

21 Why so thorough? Missing studies can have significant effect on the results of the review Publication Bias The tendency for studies with positive results to get published and studies showing no difference or negative results not getting published.

22 Thorough searching much research is never published
not all research is published in journals not all research published in journals is indexed on major databases not all research indexed on databases can be easily retrieved

23 Sources of bias in trials
Target population Sources of bias Allocation Intervention Control group group Selection bias   Exposed to Not exposed to intervention intervention Performance bias   Follow-up Follow-up Attrition bias   Outcomes Outcomes Detection bias

24 ‘Bias’ publication bias language bias
Positive results language bias Positive results more likely to be published in English language publications geographical coverage bias of journals and databases

25 Why do we search widely? Savoie et al estimated that 29.2% of items in their review were uncovered by: searching the web handsearching scanning reference lists personal communication searching specialised databases and web sites.

26 Publication bias Song et al. Publication and related biases.
Methods to dealing with publication and related biases in systematic reviews: literature searching locating unpublished studies assessment of the risk of publication and related biases detecting publication bias

27 Why search widely…. Allen & Hanburys found that: Wallace et al.
only 51% of the clinical trials relating to their respiratory products were published in journals indexed by MEDLINE, EMBASE or CINAHL (46% are in grey lit) Wallace et al. 11 of 65 trials in end stage renal disease reviews were found by searching beyond major databases. Allen and Hanburys ref from Prentice et al. Accessibility of trial data to EBM reviews. Song review: findings Research findings and dissemination profiles The empirical evidence demonstrates that studies with significant results or favourable results are more likely to be published or cited than those with non-significant or unfavourable results. Studies with significant results are often published earlier than those with non-significant results. Limited and often indirect evidence indicates only the possibility of full publication bias, outcome reporting bias, duplicate publication bias, language bias and database bias. There is some evidence concerning the existence of citation bias and media attention bias. Consequences of publication and related biases The important consequences of publication bias include the avoidable suffering of patients and the waste of limited resources. However, there is little empirical evidence relating to the impact of publication and related biases on health policy, clinical decision making and the outcome of patient management. Sources of publication bias Investigators, peer reviewers, editors and funding bodies may all be responsible for the existence of publication bias. Some evidence suggests that authors or investigators may be the main source of this bias, for not writing up or not submitting studies with null or unimportant results. However, it should be recognised that the decision to write up an article and then submit it may be affected by pressure from research sponsors and instruction from journal editors. Evidence shows that the interest of research sponsors can restrict the dissemination of research findings. The large potential variation in results obtained across similar studies that can easily be conducted and abandoned will further exacerbate the biased selection of findings for publication. Prevention of publication bias Because of their space limitations and need to maintain newsworthiness, it is unlikely that conventional paper journals can solve the problem of the selective publication of studies that produce striking results. For the purpose of reducing publication bias, peer-reviewed electronic journals that are without limitations of space are required. More importantly, editorial policy needs to be changed to accept for publication clinical trials that are based on methodological criteria only and not on the impact of their findings. Clearly, the ideal solution to publication bias is the prospective, universal registration of all studies at their inception. Although the registration of all studies cannot be realised in the near future, there are many encouraging signs that there will be more registries established as a result of initiatives from government or industry. Large-scale confirmatory studies may be an alternative in the prevention of the consequences of publication bias. Methods for reducing or detecting publication bias The methods available for dealing with publication and related biases in systematic reviews include literature searching, locating unpublished studies, assessment of the risk of publication and related biases, several methods for detecting publication bias in meta-analyses, and updating systematic reviews. The statistical methods are by nature indirect and exploratory, and often based on certain strict assumptions that can be difficult to justify in the real world. The attempt at identifying or adjusting for publication bias in a systematic review should mainly be used for the purpose of sensitivity analysis. Survey of published systematic reviews This survey indicates that literature searching was clearly inadequate in some published systematic reviews. Potential publication bias was ignored and the available methods for dealing with such bias were not used in most of these reviews. When they are used to estimate possible publication bias at the stage of literature review, the available methods were far from adequate and their usefulness was strictly limited. The problem of publication and related biases was dealt with more often in reviews containing a meta-analysis than in the narrative systematic reviews. Conclusions Although the extent, direction and impact of publication and related biases are uncertain and may vary greatly depending on circumstances, it seems reasonable to conclude that studies with significant or favourable results are more widely disseminated than those with non-significant or unfavourable results. The potential problem of publication and related biases should be taken into consideration in the field of health technology assessment. All funded or approved studies should be prospectively registered. The risk of publication bias should be assessed in all systematic reviews.

28 Why search widely.... Long lead times before publication:
publication gaps after conference presentation indexing publication lag before recorded in databases Cheng et al, 1998 8.1% of a set of conference papers achieved publication within 12 months 29% within 2 years and 40% within 5 years

29 Delay and non publication
Cheng et al, 1998 only 32% of abstracts presented at chronic fatigue conferences were subsequently published in full Petticrew et al, 1999 50.6 % of oral paper presented at the Society for Social Medicine 1996 achieved publication

30 Key factors thorough searches searching beyond published articles
citation searching Internet searching hand searching searching for ongoing research

31 Sources of Information
REVIEWS: Cochrane Database of Systematic Reviews (CDSR) DARE Cochrane Central Database of Controlled Trials (CCDCT) other sources such as Health TechnologyAssessment sources HTA ( or CCOHTA – Canadian Coordinating Office for Health Technology Assessment (has a number of publications) International Network of Agencies for Health Technology Assessment

Computer Science (INSPEC, Compendex, etc)

33 OTHER: Reference Lists & Citations
Web of Science cited reference searches Hand searching key journals Researchers in the subject area Conference Proceedings (PapersFirst, Proceedings First) Dissertations (Dissertation Abstracts International) Internet

34 Grey Literature Databases of grey literature Grey Literature Report
SIGLE System for Information on Grey Literature in Europe contains citations to reports and non-conventional (grey) literature issued informally throughout EC member countries. NetPrints -A repository of non-peer reviewed original research Organizations …. Clinical Trials

35 6. Information from included studies
Bibliographic details Study characteristics - design/methods - participants - interventions - outcome measures Study results - means, SD and/or SE - no. of events and N

36 Make sense of a review Are the results of the review valid?
B. What are the results? C. Will the results help locally?

37 A. Are the results of a review valid?
Did the review address a clearly focused issue? 2. Did the authors look for the appropriate sort of papers? 3. Do you think the important, relevant studies were included? 4. Did the review’s authors do enough to assess the quality of the included studies? 5. If the results of the review have been combined, was it reasonable to do so?

38 B. What are the results? 6. What is the overall result of the review?
‘Bottom line’ results; NNT, OR, RR, RD? 7. How precise are the results? Confidence interval, p values

39 C. Will the results help locally?
Can the results be applied to the local population? Were all important outcomes considered? Are the benefits worth the harms and costs?

40 No Questions?

Download ppt "The Bahrain Branch of the UK Cochrane Centre In Collaboration with"

Similar presentations

Ads by Google