Systematic review vs Meta-analysis Systematic review: a systematic approach to minimising bias and error Meta-analysis: a statistical analysis, which aim to produce a single estimate of a treatment effect Systematic review may or may not include Meta-analysis
Why do we need systematic review? The early 1980 s uncomplicated MI “Should pt receive a beta-blocker for secondary prevention before discharge?” Library:4 randomised controlled trials (RCT)
Beta-blocker vs placebo RCT 1.Mortality and hospital readmission is not different RCT 2. Not conclusive RCT 3. Beta-blocker not shown benefit RCT 4. Long term beta-blocker reduces the mortality and rate of re-infarction
A review in BMJ 1981 There is no clear evidence that beta- blocker improves long tem survival after MI despite almost 20 yrs of clinical trials Good enough!!
Another review in European Heart Journal 1981 “it seems perfectly reasonable to treat patients who have survived an infarction with beta-blocker”
Limitation of a single study Too small sample size false negative
Problem of Conventional review Prone to bias and error –Select only evidence support the author’s view –Not specify methodological quality of studies –Finally choose most vote ignore sample size, and design
Meta-analysis = combining all available data Attractive alternative to such large, expensive and problematic study Weight average of the result large > small trial
Meta-analysis Beta-blocker trials - MI Beta-blocker betterPlacebo better
Cumulative meta-analysis of beta-blocker trials
Cumulative meta-analysis Significant effect from 1980 onwards (OR not across 1
Benefit Estimate the overall effect Examine different result between studies (heterogeneity) Identified insufficient data
Cochrane collaboration International organisation of health care profession Promoting accessibility of systematic review Foster development of systematic review 50 collaborative review groups www.cochrane.org/cochrane/ccweb.htm
Like primary research Why- Introduction, background How-method What we found-result What it mean-discussion
Basic structure Abstract Introduction –Background –Objectives Method “treat a paper like a patient in 1 reseach” –Type of studies –Inclusion criteria type of participants –Exclusion criteria –Type of intervention –Type of outcome measures –Search strategy for identification of study –Method of analysis
Basic structure Result (special diagram) Conclusion Reference
Black square = OR, horizontal line = 95% confidence interval Area of black square = weight, diamond = combined OR with 95% CI LA better GA better Forest plot Stroke rate ‘LA vs GA in carotid sx trial’ 1966-2001
Types Systematic review of primary research –Observational studies –Diagnostic screening –RCT
3. Method “treat a paper like a patient in 1 reseach”
The process (1) Research question Writing protocol Searching Article retrieval Literature review
The process (2) Inclusion/ exclusion criteria Validity and quality of articles Data extraction/ synthesis Interpretation
The question Is local anesthesia is better than general anesthesia during carotid endarterectomy?
Writing the protocol Background Objectives Type of studies Inclusion criteria Type of participants Exclusion criteria Type of intervention Type of outcome measures Search strategy for identification of study Method of analysis Reference
Searching Medline Other database Hand searching the literature Writing to people
Getting the article Which ones to get? It takes time Libraries Inter-library Loans
Literature review You don’t have to read the whole paper yet! Translation
Validity and quality of articles Do read the paper and see what the author thought was wrong Unequal intervention/control size Hidden loss to follow up
Data extraction Read method carefully Design a form
Synthesis/ Interpretation Estimates and confidence intervals pool effect make by statistic method e.g. Peto method (fix method) give more weight effect for large study than small study (P value) Difference between studies (Heterogeneity) Chi-squared test (P value)
Small RCTs show LA is marginal lower mortality than GA
Critical Appraisal 1. Are the result valid? 2. What are the results?
1. Are the result valid? Did this review address a sensible clinical question Was the search for relavant studies detailed and exhaustive? Were selection and assessment of studies reproducible? Were the primary studies of high methodological quality?
Publication bias “A (significant) beneficial treatment effect are published, but an equal result remain unpublished” In general medical journal and public heath journal reported statistically significant 85.4% In psychological journal 95.6%
Time lag bias “Positive result will dominate the literature for several year until the negative will report later” HIV trial in USA, median time to publish of positive result 4.2 years, but negative result 6.4 years
Duplicate publication bias “ one study presents and reports several times” “ include this lead to overestimation of treatment effect” Ondersetron to prevent postoperative nausea vomitting 16 studies 3 duplicated papers Sometimes difficult to say, since not share single common authors !!!!!!
Language bias “Authors tend to report positive result in international papers, English language journal, but if negative result are published in local journal”
Outcome reporting bias In trials many outcome is recorded but only favorable finding will be reported Clinical trials by drug companies, unpublished trials gave information on adverse effect > published trials
Selection Bias Tend to happen in non Randomised controlled trials (non RCT) For example select low risk group to new treatment group
2. What are the result? Were the results similar from study to study? If yes, the credit of single estimates is OK. –Point estimates similar? –Overlapping confidence interval –Test for heterogeneity? (Chi square test) –Percentage of variability (I 2 ) good 50%