Presentation is loading. Please wait.

Presentation is loading. Please wait.

Systematic reviews – informing policy, practice and research Iain Chalmers Coordinator, James Initiative www.jameslindlibrary.org SysNet Annual Lecture,

Similar presentations


Presentation on theme: "Systematic reviews – informing policy, practice and research Iain Chalmers Coordinator, James Initiative www.jameslindlibrary.org SysNet Annual Lecture,"— Presentation transcript:

1 Systematic reviews – informing policy, practice and research Iain Chalmers Coordinator, James Initiative SysNet Annual Lecture, Cardiff University 23 April 2012

2

3 Low priority questions addressed Important outcomes not assessed Clinicians and patients not involved in setting research agendas Questions relevant to clinicians & patients? Over 50% studies designed without reference to systematic reviews of existing evidence Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation Appropriate design and methods? Over 50% of studies never published in full Biased under- reporting of studies with disappointing results Accessible full publication? Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence Unbiased and usable report? 50 % 85% Research waste = over $85 Billion / year 50 %

4 Low priority questions addressed Important outcomes not assessed Clinicians and patients not involved in setting research agendas Questions relevant to clinicians & patients? Over 50% studies designed without reference to systematic reviews of existing evidence Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation Appropriate design and methods? Over 50% of studies never published in full Biased under- reporting of studies with disappointing results Accessible full publication? Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence Unbiased and usable report? 50 % 85% Research waste = over $85 Billion / year 50 %

5 Does anyone here think that researchers should NOT review existing evidence systematically before developing treatment guidelines and planning new research?

6 Patients have suffered and died unnecessarily, and resources for health care and health research have been wasted, because existing research evidence has not been reviewed systematically.

7 JAMA 1992;268:

8

9 The human costs of failing to cumulate evidence from research scientifically “Advice on some life-saving therapies has been delayed for more than a decade, while other treatments have been recommended long after controlled research has shown them to be harmful.” Antman et al. JAMA 1992;268:240-8.

10 Oxford Textbook of Medicine, 2 nd edn, 1987 “The clinical benefits of thrombolysis … remain to be established.”

11

12 Sometimes very important advances in knowledge come from systematic reviews of existing evidence

13

14

15 "It is essential that existing sources of evidence, especially systematic reviews, are considered carefully prior to undertaking research. Research which duplicates other work unnecessarily, or which is not of sufficient quality to contribute something useful to existing knowledge, is in itself unethical." Department of Health. Research Governance Framework for Health and Social Care, 2001, para 2.3.1

16 TGN1412

17

18

19 Cumulative estimate from trials of the effect of aprotinin on perioperative blood transfusion, Aprotinin better Aprotinin worse

20 New trials of aprotinin ignored previous trials

21 A comparison of aprotinin and lysine analogues in high-risk cardiac surgery. Ferguson et al. N Engl J Med May 29;358(22): But was it the right question? Is tranexamic acid better than aprotinin?

22 Only 11 of 24 responding authors of trial reports that had been added to existing systematic reviews were even aware of the relevant reviews when they designed their new studies. *actually, the non-use *

23

24

25 20 animal studies: “The results of this review did not show convincing evidence to substantiate the decision to perform trials with nimodipine in large numbers of patients. Stroke 2001;32: STUDIES IN ANIMALS Horn J, Limburg M. Calcium antagonists for acute ischemic stroke. The Cochrane Database of Systematic Reviews, 2000 “46 trials were identified of which 28 were included (7521 patients). No effect of calcium antagonists on poor outcome at the end of follow- up (OR 1.07, 95% CI 0.97/1.18), or on death at end of follow-up (OR 1.10, 95% CI 0.98/1.24) was found.” STUDIES IN HUMANS

26 “…there is a relative scarcity of systematic reviews…” “…it would therefore be desirable to undertake further systematic reviews and meta-analyses to evaluate more fully the predictability and transferability of animal models.” 2005

27

28

29

30 Can this policy be defended on scientific, ethical, or economic grounds?

31 No subsequent contact made. Richard Hughes, personal communication, 30 Nov 2011

32

33

34

35

36

37 To reiterate Patients have suffered and died unnecessarily, and resources for health care and health research have been wasted, because existing research evidence has not been reviewed systematically.

38 Austin Bradford Hill, 1965 Four questions to which readers want answers when reading reports of research. 1. Why did you start? 2. What did you do? 3. What answer did you get? 4. And what does it mean anyway?

39

40

41 Lord Rayleigh, “One of the very few members of the higher nobility who won fame as an outstanding scientist.”

42

43 “If, as is sometimes supposed, science consisted in nothing but the laborious accumulation of facts, it would soon come to a standstill, crushed, as it were, under its own weight…The work which deserves, but I am afraid does not always receive, the most credit is that in which discovery and explanation go hand in hand, in which not only are new facts presented, but their relation to old ones is pointed out.” Lord Rayleigh, 1884

44 Things that should be done: Address uncertainties by reviewing existing evidence systematically. Ensure that new research begins and ends by referring to systematic reviews of other relevant evidence.

45 How can we expect patients and the public to trust that we have their interests at heart if we and our professional institutions continue to acquiesce in our failure to make systematic use of the results of research that they have funded?

46

47 Promote research on the effects of treatments…

48 …but only if it meets scientific and ethical principles Promote research on the effects of treatments…

49 In speaking about initiatives that arise from reviews to decide priorities Sir Paul warned that “such initiatives may attract less creative and effective scientists who simply follow where resources are being made available.” The case for greater interaction between scientists and business was also made as being crucial to rebuilding an innovation-based economy. De Havilland Alert, 1 Dec Sir Paul Nurse, President of the Royal Society, urged research funders to trust scientists to decide where public funding should be directed.

50

51

52 Finally, a challenge for you We need to be more efficient in preparing and updating systematic reviews.

53 The decline in costs of computer memory and gene sequencing Computer power: costs halve every 18 Months Gene sequencing faster! Why not systematic reviews?

54 The steps in a Systematic Review What is current time for each step? How can we make each easier/faster? –Standardize –Streamline –Computerize

55 More efficient searching Steve Pritchard Alison Weightman

56

57

58 Better abstract screening Highlight the search terms that had been used - yellow in example Highlight methodological terms (from a supplementary list NOT used in the search) – pink in example. Allowing easy marking of definite, possible, and definitely not abstracts, and comparison of these markings across two reviewers. [Sente 6]

59 Your ideas please! For any review step, do you have tips to: Standardize? Streamline? Automate? Ideas, please, to: or


Download ppt "Systematic reviews – informing policy, practice and research Iain Chalmers Coordinator, James Initiative www.jameslindlibrary.org SysNet Annual Lecture,"

Similar presentations


Ads by Google