1. Critical appraisal Systematic Review กิตติพันธุ์ ฤกษ์เกษม ภาควิชาศัลยศาสตร์ มหาวิทยาลัยเชียงใหม่

2. Systematic review vs Meta-analysis Systematic review: a systematic approach to minimising bias and error Meta-analysis: a statistical analysis, which aim to produce a single estimate of a treatment effect Systematic review may or may not include Meta-analysis

3. Why do we need systematic review? The early 1980 s uncomplicated MI “Should pt receive a beta-blocker for secondary prevention before discharge?” Library:4 randomised controlled trials (RCT)

4. Beta-blocker vs placebo RCT 1.Mortality and hospital readmission is not different RCT 2. Not conclusive RCT 3. Beta-blocker not shown benefit RCT 4. Long term beta-blocker reduces the mortality and rate of re-infarction

5. A review in BMJ 1981 There is no clear evidence that beta- blocker improves long tem survival after MI despite almost 20 yrs of clinical trials Good enough!!

6. Another review in European Heart Journal 1981 “it seems perfectly reasonable to treat patients who have survived an infarction with beta-blocker”

7. Limitation of a single study Too small sample size false negative

8. Problem of Conventional review Prone to bias and error –Select only evidence support the author’s view –Not specify methodological quality of studies –Finally choose most vote ignore sample size, and design

9. Meta-analysis = combining all available data Attractive alternative to such large, expensive and problematic study Weight average of the result large > small trial

10. Meta-analysis Beta-blocker trials - MI Beta-blocker betterPlacebo better

11. Cumulative meta-analysis of beta-blocker trials

12. Cumulative meta-analysis Significant effect from 1980 onwards (OR not across 1

13. Meta-analysis Beta-blocker trials - MI Beta-blocker betterPlacebo better Maybe Unnecessary trials

14. Benefit Estimate the overall effect Examine different result between studies (heterogeneity) Identified insufficient data

15. Cochrane collaboration International organisation of health care profession Promoting accessibility of systematic review Foster development of systematic review 50 collaborative review groups www.cochrane.org/cochrane/ccweb.htm

16. Potentials of systematic review Good Bad

17. Systematic review Basic structure and types กิตติพันธุ์ ฤกษ์เกษม ภาควิชาศัลยศาสตร์ มหาวิทยาลัยเชียงใหม่

18. 1. Basic structure

19. Like primary research Why- Introduction, background How-method What we found-result What it mean-discussion

20. Basic structure Abstract Introduction –Background –Objectives Method “treat a paper like a patient in 1  reseach” –Type of studies –Inclusion criteria type of participants –Exclusion criteria –Type of intervention –Type of outcome measures –Search strategy for identification of study –Method of analysis

21. Basic structure Result (special diagram) Conclusion Reference

22. Black square = OR, horizontal line = 95% confidence interval Area of black square = weight, diamond = combined OR with 95% CI LA better GA better Forest plot Stroke rate ‘LA vs GA in carotid sx trial’ 1966-2001

23. 2. types

24. Types Systematic review of primary research –Observational studies –Diagnostic screening –RCT

25. 3. Method “treat a paper like a patient in 1  reseach”

26. The process (1) Research question Writing protocol Searching Article retrieval Literature review

27. The process (2) Inclusion/ exclusion criteria Validity and quality of articles Data extraction/ synthesis Interpretation

28. The question Is local anesthesia is better than general anesthesia during carotid endarterectomy?

29. Writing the protocol Background Objectives Type of studies Inclusion criteria Type of participants Exclusion criteria Type of intervention Type of outcome measures Search strategy for identification of study Method of analysis Reference

30. Searching Medline Other database Hand searching the literature Writing to people

31. Getting the article Which ones to get? It takes time Libraries Inter-library Loans

32. Literature review You don’t have to read the whole paper yet! Translation

33. Validity and quality of articles Do read the paper and see what the author thought was wrong Unequal intervention/control size Hidden loss to follow up

34. Data extraction Read method carefully Design a form

35. Synthesis/ Interpretation Estimates and confidence intervals pool effect make by statistic method e.g. Peto method (fix method) give more weight effect for large study than small study (P value) Difference between studies (Heterogeneity) Chi-squared test (P value)

36. Small RCTs show LA is marginal lower mortality than GA

37. Critical Appraisal 1. Are the result valid? 2. What are the results?

38. 1. Are the result valid? Did this review address a sensible clinical question Was the search for relavant studies detailed and exhaustive? Were selection and assessment of studies reproducible? Were the primary studies of high methodological quality?

39. Publication bias “A (significant) beneficial treatment effect are published, but an equal result remain unpublished” In general medical journal and public heath journal reported statistically significant 85.4% In psychological journal 95.6%

40. Time lag bias “Positive result will dominate the literature for several year until the negative will report later” HIV trial in USA, median time to publish of positive result 4.2 years, but negative result 6.4 years

41. Duplicate publication bias “ one study presents and reports several times” “ include this lead to overestimation of treatment effect” Ondersetron to prevent postoperative nausea vomitting 16 studies 3 duplicated papers Sometimes difficult to say, since not share single common authors !!!!!!

42. Language bias “Authors tend to report positive result in international papers, English language journal, but if negative result are published in local journal”

43. Outcome reporting bias In trials many outcome is recorded but only favorable finding will be reported Clinical trials by drug companies, unpublished trials gave information on adverse effect > published trials

44. Selection Bias Tend to happen in non Randomised controlled trials (non RCT) For example select low risk group to new treatment group

45. 2. What are the result? Were the results similar from study to study? If yes, the credit of single estimates is OK. –Point estimates similar? –Overlapping confidence interval –Test for heterogeneity? (Chi square test) –Percentage of variability (I 2 ) good 50%

46. Look overlapping confidence interval Rerkasem Cochrane Database Syst Rev 2008; (4):CD000126.

47. Small RCTs show LA is marginal lower mortality than GA Rerkasem Cochrane Database Syst Rev 2008 (4):CD000126.

48. 2. What are the result? Were the results similar from study to study? What are the overall results of the reviews? How precise were the results?