1. SYSTEMATIC REVIEWS AND META-ANALYSIS

2. Objectives Define systematic review and meta-analysis Know how to
access
appraise
interpret
the results of a systematic review

3. Reasons to use reviews Sheer volume of literature*
Save time doing exhaustive literature researches
*Wyatt: about 25,000 biomedical journals in print worldwide.
This number is rising by 4% pa - doubling time of 19 yrs

4. Review
Any attempt to synthesise results and conclusions of 2 or more publications on a given topic
e.g. editorials, working papers
Problems
Retrieval bias and publication bias
Were all studies identified?
Do not see how conclusions arrived at
Reviewers may be biased / conflict of interest
Languages excluded?

5. Were all studies that met the inclusion
criteria for the review identified successfully?
If not, were the results of the sample of studies
included in the review representative of the
results of all eligible studies?
Of controlled trials identified in five separate subject
areas within perinatal medicine, between 20 and 50
percent were identified using MEDLINE, compared
with between 85 and 100 percent using the Oxford
Database of Perinatal Trials.
Chalmers et al. 1989

6. Retrieval bias What can be done about it? How can it be prevented?
Systematic and comprehensive search for eligible reports
How can it be prevented?
Use of structured abstracts by investigators
Improvement, extension and further development of
Bibliographic retrieval systems
Explicit search criteria

7. Publication bias
A tendency among investigators, peer reviewers and journal editors to allow the direction and statistical significance of research findings to influence decisions regarding submission and acceptance for publication

8. Publication bias Examples
Analysis of relative risks derived from studies examining the relationship between passive smoking and lung cancer suggests that studies in which no relationship has been detected may remain unpublished Vandenbrouke, Br Med J 1988;296:
However, estimates of the effect of these suggests that they would have little impact on the weight of evidence against passive smoking, 2004
A survey of authors or published reports of randomized trials revealed that between a quarter and a fifth of the trials that they have ever conducted had never been published, and that trials in which a new treatment had been found to be a superior to a standard treatment were more likely than others to have been published.
Dickersin et al.Contr Clin Trials1987;8:

9. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ  2003;326:  (31 May), Joel Lexchin, Lisa A Bero, Benjamin Djulbegovic, Otavio Clark
Results
30 studies were included (1966 – 2002). Research funded by drug companies was less likely to be published than research funded by other sources. Studies sponsored by pharmaceutical companies were more likely to have outcomes favouring the sponsor than were studies with other sponsors (odds ratio 4.05; 95% confidence interval 2.98 to 5.51; 18 comparisons). None of the 13 studies that analysed methods reported that studies funded by industry was of poorer quality.

10. Publication bias What can be done about it? Grey literature
Contact authors

11. Pitfalls of reviews
“Current medical reviews do not routinely use scientific methods to identify, assess and synthesise information.” Mulrow, 1987
50 reviews in 4 major journals,
no statement of methods 49
summary inappropriate 47

12. How can the situation improve?
Better reviews (high quality, more relevant)
Cochrane collaboration
Systematic reviews
Improved access to reviews
CCPC, Databases, Effectiveness Bulletins
Readers more skilled in making sense of reviews
CASP

13. What is a systematic review?
A review in which evidence on a topic has been
systematically identified and summarised according
to predetermined criteria.
Specific clinical questions
Predefined explicit methodology
Reproducible
Usually review of RCTs

14. Meta-analysis - statistical principles
The use of statistical methods to summarise the results of independent studies into a single estimate giving more weight to results from larger studies.
No direct comparison of patients
Summary statistics are calculated for each trial
Individual estimates are pooled - overall pooled estimate (if appropriate)
Gives more precise estimate of effect size
Interpretation / odds ratios

15. Types of review
Reviews
Systematic reviews
Meta-analysis

16. Why use systematic reviews?
Volume of literature
Provide a basis for rational decision making
Are health care effects consistent?
Limit bias and reduce random error
Provides more reliable results
Required for ethics committees, funding agencies

17. Aim: To help people make well-informed decisions about health care by preparing, maintaining and promoting accessibility of systematic reviews of the effects of health care interventions.

18. 6 principles of Cochrane
Collaboration
5000 people in 50 countries
Building on peoples’ existing enthusiasm and interests
50 Cochrane Review Groups - focus on particular areas of health
Minimising duplication effort
Avoidance of bias
Keeping up to date
Ensuring access

19. Format for a (Cochrane) review
Contact Cochrane
Develop a protocol
Formulate the problem
Locate and select studies
Critical appraisal of studies
Score for selection, attrition, performance, detection, blinding biases
Collecting data
Analysing and presenting results
Interpreting results
Improving and updating reviews

20. What is a Cochrane Review. http://www. cochrane. org/reviews/revstruc
Cochrane Reviews investigate the effects of interventions for prevention, treatment and rehabilitation in a healthcare setting. They are designed to facilitate the choices that doctors, patients, policy makers and others face in health care. Most Cochrane Reviews are based on randomized controlled trials, but other types of evidence may also be taken into account, if appropriate. Cochrane reviews have the following general features:

21. General features of a Cochrane review
Structured format
Detailed methods section
The quality of clinical studies to be incorporated into a review is carefully considered, using predefined criteria.
A thorough and systematic search strategy, which includes searches for unpublished and non-English records
If the data collected in a review are of sufficient quality and similar enough, they are summarised statistically in a meta-analysis,
generally provides a better overall estimate of a clinical effect than the results from individual studies.
Reviews aim to be relatively easy to understand for non-experts
Multinational editorial teams try to ensure that a review is applicable in different parts of the world
Reviews are updatable.
Results from newly completed or identified clinical trials can be incorporated into the review after publication. Additionally, readers can send in comments and criticisms to any review, and reviews may be changed accordingly to improve their quality.

22. Systematic review protocol
State objectives and eligibility criteria
Identify potentially eligible studies
Apply eligibility criteria
Refine protocol
Publish protocol

23. Elements of a systematic review
Define the clinical question
Identify all completed studies, un/published
Select the studies that meet scientific validity criteria
Look for evidence of bias
Describe the scientific quality of the studies
Assess if quality systematically related to results
Describe studies with a forest plot
Assess if similar enough to justify combining results
Calculate summary measure of effect & CI
Fletcher & Fletcher T12.1

24. How can we appraise a review?
How can we systematically appraise a review?
What questions should we ask?

25. Three basic types of question... Is it trustworthy? – Validity
10 questions to appraise a review Critical Appraisal Skills Programme (CASP)
Three basic types of question...
Is it trustworthy? – Validity
Screening questions
Detailed questions on methodology
What does it say? – Results
Will it help? – Relevance

26. Is it trustworthy? – Validity
Screening questions
Did review address a clearly focused issue?
Did authors review right type of study?
Detailed questions on methodology
Were all important relevant studies included?
Did reviewers do enough to assess the quality of the included studies?
Randomisation system, scoring system, >1 assessor
If meta-analysis performed, was it reasonable to combine the results?

27. What does it say - Results
Rates
Relative and absolute risk, numbers needed to treat + confidence intervals
Clinical significance
Statistical significance

28. Will it help? – Relevance (What service providers should ask)
Is my patient...
Are the interventions available to my patient...
Are the outcomes relevant to my patient...
.....sufficiently different from people in the review to allow me to consider the findings inapplicable?

29. Design literature search for: Quantitative systematic review of randomised controlled trials comparing antibiotic with placebo for acute cough in adults. (BMJ 1998;316:906-10) Fahey T. et al.
Aim:
To establish whether antibiotics are effective in the treatment of acute cough in the community

30. Literature search Medical subject headings (MeSH) Databases Language?
Study type
Patients

31. Literature search Medical subject headings (MeSH) Databases
Cough, bronchitis, sputum, respiratory tract infections, chest infection
Databases
Medline, EMBASE, Science Citation Index, Cochrane Controlled Trials Register
Contacted authors - ?know of unpublished trials
UK drug companies ?unpublished trials
Language - not just English

32. Literature search Inclusion/exclusion criteria
Study type
Prospective trials, formal or quasi-randomisation
Placebo controlled
Patients
>12 years
Family practice clinic, community based o/px dept., hospital o/px dept.
Acute cough, with/out sputum, no antibiotic in preceding week
COAD excluded

33. Get class to interpret this table

34. Conclusions? Treatment with antibiotic Benefits
Does not affect resolution of cough or alter course of illness
Benefits
Marginal for most patients with acute cough
May be outweighed by side effects (RR=1.51, CI 0.86 to 2.64)