Presentation is loading. Please wait.

Presentation is loading. Please wait.

Improving the internal validity of experiments in focal ischaemia Malcolm Macleod Centre for Clinical Brain Sciences, University of Edinburgh.

There are copies: 1
Preventing introduction of bias at the bench: from randomizing to experimental results meta-analysis Malcolm Macleod Centre for Clinical Brain Sciences,

Similar presentations


Presentation on theme: "Improving the internal validity of experiments in focal ischaemia Malcolm Macleod Centre for Clinical Brain Sciences, University of Edinburgh."— Presentation transcript:

1 Improving the internal validity of experiments in focal ischaemia Malcolm Macleod Centre for Clinical Brain Sciences, University of Edinburgh

2 interventions in experimental stroke OCollins et al Ann Neurol 2006

3 interventions in experimental stroke Tested in focal ischaemia OCollins et al Ann Neurol 2006

4 interventions in experimental stroke Effective in focal ischaemia OCollins et al Ann Neurol 2006

5 interventions in experimental stroke Tested in clinical trial OCollins et al Ann Neurol 2006

6 interventions in experimental stroke Effective in clinical trial OCollins et al Ann Neurol 2006

7 Whats my problem? I want to improve the outcome for my patients with stroke To get that, I want to conduct high quality clinical trials of interventions which have a reasonable chance of actually working in humans But which of the remaining 929 interventions should I choose?

8 Its not just my problem …

9 …you will meet with several observations and experiments which, though communicated for true by candid authors or undistrusted eye-witnesses, or perhaps recommended by your own experience, may, upon further trial, disappoint your expectation, either not at all succeeding, or at least varying much from what you expected Robert Boyle (1693) Concerning the Unsuccessfulness of Experiments

10 One which describes some biological truth in the system being studied Internal validity: the extent to which an experiment accurately describes what happened in that model system Can be inferred by extent of reporting of measures to avoid common biases What is a Valid Experiment?

11 One which considers all available supporting animal data One which considers the likelihood of publication bias One which tests a drug under circumstances similar to those in which efficacy has been demonstrated in animal models What is a Valid Translational strategy?

12 Potential sources of bias in animal studies Internal validity ProblemSolution Selection BiasRandomisation Performance BiasAllocation Concealment Detection BiasBlinded outcome assessment Attrition biasReporting drop-outs/ ITT analysis False positive report biasAdequate sample sizes After Crossley et al, 2008, Wacholder, 2004

13 Internal validity Dopamine Agonists in models of PD Ferguson et al, in draft

14 Internal validity Dopamine Agonists in models of PD Ferguson et al, in draft

15 Internal validity Dopamine Agonists in models of PD Ferguson et al, in draft

16 Internal Validity Randomisation and blinding in studies of hypothermia in experimental stroke van der Worp et al Brain 2007 Randomisation Yes No Blinded outcome assessment Yes No Efficacy 47%39% 47%37% Efficacy

17 Stem cells in experimental stroke Lees et al, in draft

18 Infarct Volume Internal Validity Randomisation, allocation concealment and blinding in studies of Stem cells in experimental stroke Neurobehavioural score Lees et al, in draft

19 Internal Validity NXY-059 Macleod et al, 2008

20 Internal Validity Attrition bias

21 Internal Validity False positive reporting bias The positive predictive value of any test result depends on –p ( α) –Power (1-ß) –Pre-test probability of a positive result after Wacholder, 2004

22 Internal Validity False positive reporting bias The positive predictive value of any test result depends on –p ( α) (0.05) –Power (1-ß) (0.30) –Pre-test probability of a positive result (0.50) Positive predictive value = 0.67 i.e. only 2 out of 3 statistically positive studies are truly positive after Wacholder, 2004

23 Chances that data from any given animal will be non-contributory Number of animalsPower% animals wasted 418.6%81.4% 832.3%67.7% %43.6% %14.9% assume simple two group experiment seeking 30% reduction in infarct volume, observed SD 40% of control infarct volume

24 Chances of wasting an animal

25 BetterWorse Precision All outcomes – 29 publications – 109 experiments – 1596 animals – Improved outcome by 31% (27-35%) External Validity Publication Bias for FK506 Macleod et al, JCBFM 2005

26 External Validity Hypertension in studies of NXY-059 in experimental stroke Macleod et al, Stroke in press Infarct volume: – 9 publications – 29 experiments – 408 animals – 44% (35-53%) improvement Hypertension: – 7% of animal studies – 77% of patients in the (neutral) SAINT II study

27 External Validity Hypertension in studies of tPA in experimental stroke Perel et al BMJ 2007 Comorbidity Normal BP Efficacy -2% 25% Infarct Volume: –113 publications –212 experiments –3301 animals –Improved outcome by 24% (20-28) Hypertension: –9% of animal studies –Specifically exclusion criterion in (positive) NINDS study

28 Quality of Translation tPA and tirilazad Both appear to work in animals tPA works in humans but tirilazad doesnt Time to treatment: tPA: –Animals– median 90 minutes –Clinical trial– median 90 minutes Time to treatment: tirilazad –Animals– median 10 minutes –Clinical trial- >3 hrs for >75% of patients Sena et al, Stroke 2007; Perel et al BMJ 2007

29 Chose your patients – tPA: Effect of time to treatment on efficacy Perel et al BMJ 2007; Lancet 2004

30 Publication bias RandomisationCo-morbidity bias Reported efficacy How much efficacy is left? 26% 32% 20% 5%

31 Animal Studies Systematic Review And Meta-analysis how powerful is the treatment? what is the quality of evidence? what is the range of evidence? is there evidence of a publication bias? What are the conditions of maximum efficacy? Clinical Trial Summarising data from animal experiments

32 Resources and acknowledgements Chief Scientist Office, Scotland Emily Sena, Evie Ferguson, Jen Lees, Hanna Vesterinen David Howells, Bart van der Worp, Uli Dirnagl, Philip Bath


Download ppt "Improving the internal validity of experiments in focal ischaemia Malcolm Macleod Centre for Clinical Brain Sciences, University of Edinburgh."

Similar presentations


Ads by Google