Presentation is loading. Please wait.

Presentation is loading. Please wait.

CAMARADES: Bringing evidence to translational medicine The failure to translate the basic science into therapy is due primarily to inadequacies in the.

Published byJenna Henry Modified over 10 years ago

Similar presentations

Presentation on theme: "CAMARADES: Bringing evidence to translational medicine The failure to translate the basic science into therapy is due primarily to inadequacies in the."— Presentation transcript:

1 CAMARADES: Bringing evidence to translational medicine The failure to translate the basic science into therapy is due primarily to inadequacies in the preclinical (animal) data

2 CAMARADES: Bringing evidence to translational medicine …or

3 CAMARADES: Bringing evidence to translational medicine The quality of most animal studies is so shockingly poor that their results are an unreliable indicator of what happens in animals let alone what might happen in humans

4 CAMARADES: Bringing evidence to translational medicine 1026 1026 interventions in experimental stroke

5 CAMARADES: Bringing evidence to translational medicine 1026 603 1026 interventions in experimental stroke Tested in focal ischaemia

6 CAMARADES: Bringing evidence to translational medicine 1026 883 374 1026 interventions in experimental stroke Effective in focal ischaemia

7 CAMARADES: Bringing evidence to translational medicine 1026 883 550 9718 1026 interventions in experimental stroke Tested in clinical trial

8 CAMARADES: Bringing evidence to translational medicine 1026 883 550 9717 13 1026 interventions in experimental stroke Effective in clinical trial

9 CAMARADES: Bringing evidence to translational medicine Where are we going wrong? Are animal experiments falsely positive? Have clinical trials tested the conditions of maximum efficacy? … and what, if anything, does this mean for models of other diseases?

10 CAMARADES: Bringing evidence to translational medicine 10 -120 M10 -60 M

11 CAMARADES: Bringing evidence to translational medicine Animal data in stroke There are huge amounts of often confusing data Systematic review can help to make sense of it If you select extreme bits of the evidence you can prove either harm or substantial benefit However, if you have a precise and highly significant overall effect, then it is probably real Hypothermia: a systematic search identified 277 experiments in 3353 animals Better Worse

12 CAMARADES: Bringing evidence to translational medicine

13 Where are we going wrong? Are animal experiments falsely positive? Are clinical trials falsely negative? Do animal studies not model human disease with sufficient fidelity to be useful?

14 CAMARADES: Bringing evidence to translational medicine Potential sources of bias in animal studies Internal validity External validity –Publication bias –Are the models we use good models? Co-morbidities ProblemSolution Selection BiasRandomisation Performance BiasAllocation Concealment Detection BiasBlinded outcome assessment Attrition biasReporting drop-outs/ ITT analysis

15 CAMARADES: Bringing evidence to translational medicine Internal Validity Hypothermia in experimental stroke Infarct Volume –101 publications –222 experiments –3256 animals –Improved outcome by 43.5% (40.1-47.0)

16 CAMARADES: Bringing evidence to translational medicine Internal Validity Randomisation and blinding in studies of hypothermia in experimental stroke Randomisation Yes No Blinded outcome assessment Yes No Efficacy 47%39% 47%37% Efficacy

17 CAMARADES: Bringing evidence to translational medicine Internal Validity Stem Cell based therapies Infarct Volume –54 publications –127 experiments –2012 animals –Improved outcome by 28.9% (24.8-33.0) Neurobehavioural score: –72 publications –111 experiments –1876 animals –Improved outcome by 34.4% (29.5-39.2)

18 CAMARADES: Bringing evidence to translational medicine Randomisation Stem Cell based therapies

19 CAMARADES: Bringing evidence to translational medicine Blinded outcome assessment Stem Cell based therapies

20 CAMARADES: Bringing evidence to translational medicine Internal Validity NXY-059 Candidate neuroprotective drug unsuccessful in clinical trial Infarct Volume –11 publications –29 experiments –408 animals –Improved outcome by 44% (35-53% )

21 CAMARADES: Bringing evidence to translational medicine Internal Validity NXY-059

22 CAMARADES: Bringing evidence to translational medicine Reported Efficacy36% Corrected Efficacy<0%

23 CAMARADES: Bringing evidence to translational medicine The File Drawer problem Publication bias 0 worsebetter

24 CAMARADES: Bringing evidence to translational medicine BetterWorse Precision All outcomes – 29 publications – 109 experiments – 1596 animals – Improved outcome by 31% (27-35%) External Validity Publication Bias for FK506

25 CAMARADES: Bringing evidence to translational medicine Publication bias - gCSF England T et al 2009

26 CAMARADES: Bringing evidence to translational medicine Publication bias in experimental stroke Only 11/525 publications (2.2%) reported no significant treatment effects Trim and Fill suggested ~16% (214/1573) of experiments remain unpublished Best estimate of magnitude of problem –Observed efficacy31.3% (29.7-32.8) –Adjusted efficacy 23.8% (22.2-25.5)

27 CAMARADES: Bringing evidence to translational medicine External Validity Hypertension in studies of tPA in experimental stroke Comorbidity Normal BP Efficacy -2% 25% Infarct Volume – 113 publications – 212 experiments – 3301 animals – Improved outcome by 24% (20-28)

28 CAMARADES: Bringing evidence to translational medicine External Validity Hypertension in studies of NXY-059 7% of studies used animals with hypertension 77% of patients in SAINT II had a history of hypertension at study entry

29 CAMARADES: Bringing evidence to translational medicine Summary Certain aspects of the design of animal experiments probably do lead to the over- statement of neuroprotective efficacy A substantial publication bias is present Neuroprotective efficacy may be substantially lower in animals with relevant co-morbidities

30 CAMARADES: Bringing evidence to translational medicine Publication bias RandomisationCo-morbidity bias How much efficacy is left?

31 CAMARADES: Bringing evidence to translational medicine …you will meet with several observations and experiments which, though communicated for true by candid authors or undistrusted eye-witnesses, or perhaps recommended by your own experience, may, upon further trial, disappoint your expectation, either not at all succeeding, or at least varying much from what you expected Robert Boyle (1693), Concerning the Unsuccessfulness of Experiments

32 CAMARADES: Bringing evidence to translational medicine A toolkit for effective translation Clear, rigorous SOPs for all aspects of experimental design On-line tools for –Sample size calculation –Random allocation to group Development of experimental methods and funding streams to support multi-centre animal studies Adoption of CONSORT statement for animal stroke studies

33 CAMARADES: Bringing evidence to translational medicine

34 Chances that data from any given animal will be non-contributory Number of animalsPower% animals wasted 418.6%81.4% 832.3%67.7% 1656.4%43.6% 3285.1%14.9% assume simple two group experiment seeking 30% reduction in infarct volume, observed SD 40% of control infarct volume

35 CAMARADES: Bringing evidence to translational medicine

36 Current performance against key quality items Randomisation Blinded Outcome Assessment Sample Size calculation Stroke36%29%3%

37 CAMARADES: Bringing evidence to translational medicine How does stroke compare? Randomisation Blinded Outcome Assessment Sample Size calculation Stroke36%29%3% MND31%20%<1% AD15%25%0% PD12%15%0% EAE8%15%<1% Glioma14%0%

38 CAMARADES: Bringing evidence to translational medicine Systematic review - PD

39 CAMARADES: Bringing evidence to translational medicine Internal validity in PD models Blinded outcome assessmentComposite quality

40 CAMARADES: Bringing evidence to translational medicine Modelling MS

41 CAMARADES: Bringing evidence to translational medicine External validity - MS

42 CAMARADES: Bringing evidence to translational medicine Animal Studies Systematic Review And Meta-analysis how powerful is the treatment? what is the quality of evidence? what is the range of evidence? is there evidence of a publication bias? What are the conditions of maximum efficacy? Clinical Trial Summarising data from animal experiments

43 CAMARADES: Bringing evidence to translational medicine CAMARADES Bringing evidence to translational medicine Disease modelInterventionsPublicationsExperimentsAnimals Focal cerebral ischaemia 17556143920690 Intracerebral Haemorrhage 62974073647 Experimental Allergic Encephalomyelitis 36 (1717)123 (1152)4387224 Transgenic models of AD 207612179422000 Parkinsons Disease28573032245 Spinal Cord Injury34693313596 Total2543471259402

44 CAMARADES: Bringing evidence to translational medicine Estimates of affinity at cannabinoid receptors McPartland et al 2007

45 CAMARADES: Bringing evidence to translational medicine Immunisation EAE Th17 Th1 CD4+ -=+ -6 =3 +61 EAE Th1 -=+ -111 =1 +22 EAE Th17 What causes EAE?

46 CAMARADES: Bringing evidence to translational medicine CAMARADES Bringing evidence to translational medicine Metrics of Research Quality –Does Journal Impact Factor reflect study quality? JIF = 3.7 + 2.4 (Conflict of Interest statement) + 1.2 (Blinded Induction of ischaemia) 465 publications: adjusted r 2 = 0.06

47 CAMARADES: Bringing evidence to translational medicine CAMARADES Bringing evidence to translational medicine Most animal studies cant even tell you what happened in the animals –Underpowered –Poor internal validity –Publication bias Functional (Neurobehavioural) outcome is not so different from structural (infarct size) outcome Judging from journal impact factors, scientists cant tell the difference between high quality studies and low quality studies

48 CAMARADES: Bringing evidence to translational medicine Un Canard

49 CAMARADES: Bringing evidence to translational medicine

50 Un canard mort

51 CAMARADES: Bringing evidence to translational medicine

52 Quality of Translation tPA and tirilazad Both appear to work in animals tPA works in humans but tirilazad doesnt Time to treatment: tPA: –Animals– median 90 minutes –Clinical trial– median 90 minutes Time to treatment: tirilazad –Animals– median 10 minutes –Clinical trial- >3 hrs for >75% of patients

53 CAMARADES: Bringing evidence to translational medicine tPA: Effect of time to treatment on efficacy

54 CAMARADES: Bringing evidence to translational medicine Animal Studies Clinical Trial Systematic Review And Meta-analysis

55 CAMARADES: Bringing evidence to translational medicine Whatever happened to NXY-059?

56 CAMARADES: Bringing evidence to translational medicine Astra Zeneca Share Price

57 CAMARADES: Bringing evidence to translational medicine What is a neuroprotective drug worth? Current market capitalisation –£31.98bn Current Market Price –£21.95 – approx 15 million shares Price fell from £35.29 to £31.52 in first week – Market valuation of neuroprotective efficacy is approximately £5.5bn

58 CAMARADES: Bringing evidence to translational medicine What forces drive bias?

Download ppt "CAMARADES: Bringing evidence to translational medicine The failure to translate the basic science into therapy is due primarily to inadequacies in the."

Similar presentations

Study Quality and Publication Bias in Experimental Stroke

Study Quality and Publication Bias in Experimental Stroke
Numbers Treasure Hunt Following each question, click on the answer. If correct, the next page will load with a graphic first – these can be used to check.

Numbers Treasure Hunt Following each question, click on the answer. If correct, the next page will load with a graphic first – these can be used to check.
2 Casa 15m Perspectiva Lateral Izquierda.

2 Casa 15m Perspectiva Lateral Izquierda.
1 A B C 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52.

1 A B C
AP STUDY SESSION 2.

AP STUDY SESSION 2.
1

1
Worksheets.

Worksheets.
Copyright © 2003 Pearson Education, Inc. Slide 1 Computer Systems Organization & Architecture Chapters 8-12 John D. Carpinelli.

Copyright © 2003 Pearson Education, Inc. Slide 1 Computer Systems Organization & Architecture Chapters 8-12 John D. Carpinelli.
STATISTICS HYPOTHESES TEST (I)

STATISTICS HYPOTHESES TEST (I)
David Burdett May 11, 2004 Package Binding for WS CDL.

David Burdett May 11, 2004 Package Binding for WS CDL.
CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Studies of Neurological Diseases Emily S Sena,

CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Studies of Neurological Diseases Emily S Sena,
Summarising what we already know – the pivotal role of systematic reviews Malcolm Macleod.

Summarising what we already know – the pivotal role of systematic reviews Malcolm Macleod.
Structural and functional outcomes in animal models of stroke: What do they measure? Malcolm Macleod University of Edinburgh.

Structural and functional outcomes in animal models of stroke: What do they measure? Malcolm Macleod University of Edinburgh.
Summarising the evidence from animal models of neurological disease: Publication bias, poor internal validity, and (perhaps) some efficacy Malcolm Macleod.

Summarising the evidence from animal models of neurological disease: Publication bias, poor internal validity, and (perhaps) some efficacy Malcolm Macleod.
CAMARADES: Bringing evidence to translational medicine Stanford visit 8 Feb 2011.

CAMARADES: Bringing evidence to translational medicine Stanford visit 8 Feb 2011.
Modelling Stroke in the Laboratory - Separating Fact from Artefact The impact of sources of bias in animal models of neurological disease, and what we.

Modelling Stroke in the Laboratory - Separating Fact from Artefact The impact of sources of bias in animal models of neurological disease, and what we.
Summarising the evidence from animal models of neurological disease: Are there any babies in the bathwater? Malcolm Macleod University of Edinburgh.

Summarising the evidence from animal models of neurological disease: Are there any babies in the bathwater? Malcolm Macleod University of Edinburgh.
Improving the internal validity of experiments in focal ischaemia

Improving the internal validity of experiments in focal ischaemia
Preventing introduction of bias at the bench: from randomizing to experimental results meta-analysis Malcolm Macleod Centre for Clinical Brain Sciences,

Preventing introduction of bias at the bench: from randomizing to experimental results meta-analysis Malcolm Macleod Centre for Clinical Brain Sciences,
Summarising the evidence from animal models of neurological disease: Publication bias, poor internal validity, and (perhaps) some efficacy Malcolm Macleod.

Summarising the evidence from animal models of neurological disease: Publication bias, poor internal validity, and (perhaps) some efficacy Malcolm Macleod.

Similar presentations

Ads by Google