Presentation is loading. Please wait.

Presentation is loading. Please wait.

Improving the internal validity of experiments in focal ischaemia

Published byElijah McDowell Modified over 10 years ago

There are copies: 1
Preventing introduction of bias at the bench: from randomizing to experimental results meta-analysis Malcolm Macleod Centre for Clinical Brain Sciences,
Preventing introduction of bias at the bench: from randomizing to experimental results meta-analysis Malcolm Macleod Centre for Clinical Brain Sciences,

Similar presentations

Presentation on theme: "Improving the internal validity of experiments in focal ischaemia"— Presentation transcript:

1 Improving the internal validity of experiments in focal ischaemia
Malcolm Macleod Centre for Clinical Brain Sciences, University of Edinburgh

2 1026 interventions in experimental stroke
O’Collins et al Ann Neurol 2006

3 1026 interventions in experimental stroke
603 O’Collins et al Ann Neurol 2006 Tested in focal ischaemia

4 1026 interventions in experimental stroke
883 374 O’Collins et al Ann Neurol 2006 Effective in focal ischaemia

5 1026 interventions in experimental stroke
883 550 97 18 O’Collins et al Ann Neurol 2006 Tested in clinical trial

6 1026 interventions in experimental stroke
883 1 3 97 17 550 O’Collins et al Ann Neurol 2006 Effective in clinical trial

7 What’s my problem? I want to improve the outcome for my patients with stroke To get that, I want to conduct high quality clinical trials of interventions which have a reasonable chance of actually working in humans But which of the remaining 929 interventions should I choose?

8 It’s not just my problem …

9 “…you will meet with several observations and experiments which, though communicated for true by candid authors or undistrusted eye-witnesses, or perhaps recommended by your own experience, may, upon further trial, disappoint your expectation, either not at all succeeding, or at least varying much from what you expected” Robert Boyle (1693) Concerning the Unsuccessfulness of Experiments

10 What is a Valid Experiment?
One which describes some biological truth in the system being studied Internal validity: the extent to which an experiment accurately describes what happened in that model system Can be inferred by extent of reporting of measures to avoid common biases

11 What is a Valid Translational strategy?
One which considers all available supporting animal data One which considers the likelihood of publication bias One which tests a drug under circumstances similar to those in which efficacy has been demonstrated in animal models

12 Potential sources of bias in animal studies
Internal validity Problem Solution Selection Bias Randomisation Performance Bias Allocation Concealment Detection Bias Blinded outcome assessment Attrition bias Reporting drop-outs/ ITT analysis False positive report bias Adequate sample sizes After Crossley et al, 2008, Wacholder, 2004 Are animal experiments falsely positive? Are clinical trials falsely negative? Do animal studies not model human disease with sufficient fidelity to be useful?

13 Internal validity Dopamine Agonists in models of PD
Ferguson et al, in draft

14 Internal validity Dopamine Agonists in models of PD
Ferguson et al, in draft

15 Internal validity Dopamine Agonists in models of PD
Ferguson et al, in draft

16 Blinded outcome assessment
Internal Validity Randomisation and blinding in studies of hypothermia in experimental stroke Randomisation Blinded outcome assessment Yes No Efficacy è 47% 39% 47% 37% Efficacy è van der Worp et al Brain 2007 Are animal experiments falsely positive? Are clinical trials falsely negative? Do animal studies not model human disease with sufficient fidelity to be useful? Yes No

17 Stem cells in experimental stroke
Lees et al, in draft

18 Neurobehavioural score
Internal Validity Randomisation, allocation concealment and blinding in studies of Stem cells in experimental stroke Infarct Volume Neurobehavioural score Lees et al, in draft

19 Internal Validity NXY-059
Macleod et al, 2008

20 Internal Validity Attrition bias

21 Internal Validity False positive reporting bias
The positive predictive value of any test result depends on p (α) Power (1-ß) Pre-test probability of a positive result after Wacholder, 2004

22 Internal Validity False positive reporting bias
The positive predictive value of any test result depends on p (α) (0.05) Power (1-ß) (0.30) Pre-test probability of a positive result (0.50) after Wacholder, 2004 Positive predictive value = 0.67 i.e. only 2 out of 3 statistically positive studies are truly positive

23 Chances that data from any given animal will be non-contributory
assume simple two group experiment seeking 30% reduction in infarct volume, observed SD 40% of control infarct volume Number of animals Power % animals wasted 4 18.6% 81.4% 8 32.3% 67.7% 16 56.4% 43.6% 32 85.1% 14.9%

24 Chances of wasting an animal

25 External Validity Publication Bias for FK506
All outcomes 29 publications 109 experiments 1596 animals Improved outcome by 31% (27-35%) Precision Macleod et al, JCBFM 2005 Worse Better

26 External Validity Hypertension in studies of NXY-059 in experimental stroke
Infarct volume: 9 publications 29 experiments 408 animals 44% (35-53%) improvement Hypertension: 7% of animal studies 77% of patients in the (neutral) SAINT II study Macleod et al, Stroke in press

27 External Validity Hypertension in studies of tPA in experimental stroke
Infarct Volume: 113 publications 212 experiments 3301 animals Improved outcome by 24% (20-28) Hypertension: 9% of animal studies Specifically exclusion criterion in (positive) NINDS study Efficacy è 25% -2% “Normal” ÝBP Perel et al BMJ 2007 Comorbidity

28 Quality of Translation tPA and tirilazad
Both appear to work in animals tPA works in humans but tirilazad doesn’t Time to treatment: tPA: Animals – median 90 minutes Clinical trial – median 90 minutes Time to treatment: tirilazad Animals – median 10 minutes Clinical trial - >3 hrs for >75% of patients Sena et al, Stroke 2007; Perel et al BMJ 2007 Are animal experiments falsely positive? Are clinical trials falsely negative? Do animal studies not model human disease with sufficient fidelity to be useful?

29 Chose your patients – tPA: Effect of time to treatment on efficacy
Perel et al BMJ 2007; Lancet 2004 11% of all patients treatment initiated by 90 minutes 33% of sub-3hr patients treatment initiated by 90 minutes

30 How much efficacy is left?
Reported efficacy Publication bias Randomisation Co-morbidity bias 26% 20% 5% 32%

31 Summarising data from animal experiments
Studies Systematic Review And Meta-analysis Clinical Trial how powerful is the treatment? what is the quality of evidence? what is the range of evidence? is there evidence of a publication bias? What are the conditions of maximum efficacy?

32 Resources and acknowledgements
Chief Scientist Office, Scotland Emily Sena, Evie Ferguson, Jen Lees, Hanna Vesterinen David Howells, Bart van der Worp, Uli Dirnagl, Philip Bath

Download ppt "Improving the internal validity of experiments in focal ischaemia"

Similar presentations

Study Quality and Publication Bias in Experimental Stroke

Study Quality and Publication Bias in Experimental Stroke
Numbers Treasure Hunt Following each question, click on the answer. If correct, the next page will load with a graphic first – these can be used to check.

Numbers Treasure Hunt Following each question, click on the answer. If correct, the next page will load with a graphic first – these can be used to check.
Jack Jedwab Association for Canadian Studies September 27 th, 2008 Canadian Post Olympic Survey.

Jack Jedwab Association for Canadian Studies September 27 th, 2008 Canadian Post Olympic Survey.
Lecture 8: Hypothesis Testing

Lecture 8: Hypothesis Testing
AP STUDY SESSION 2.

AP STUDY SESSION 2.
1

1
& dding ubtracting ractions.

& dding ubtracting ractions.
STATISTICS HYPOTHESES TEST (I)

STATISTICS HYPOTHESES TEST (I)
Design of Dose Response Clinical Trials

Design of Dose Response Clinical Trials
The Application of Propensity Score Analysis to Non-randomized Medical Device Clinical Studies: A Regulatory Perspective Lilly Yue, Ph.D.* CDRH, FDA,

The Application of Propensity Score Analysis to Non-randomized Medical Device Clinical Studies: A Regulatory Perspective Lilly Yue, Ph.D.* CDRH, FDA,
OPTN Modifications to Heart Allocation Policy Implemented July 12, 2006 Changed the allocation order for medically urgent (Status 1A and 1B) patients Policy.

OPTN Modifications to Heart Allocation Policy Implemented July 12, 2006 Changed the allocation order for medically urgent (Status 1A and 1B) patients Policy.
David Burdett May 11, 2004 Package Binding for WS CDL.

David Burdett May 11, 2004 Package Binding for WS CDL.
CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Studies of Neurological Diseases Emily S Sena,

CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Studies of Neurological Diseases Emily S Sena,
Summarising what we already know – the pivotal role of systematic reviews Malcolm Macleod.

Summarising what we already know – the pivotal role of systematic reviews Malcolm Macleod.
Structural and functional outcomes in animal models of stroke: What do they measure? Malcolm Macleod University of Edinburgh.

Structural and functional outcomes in animal models of stroke: What do they measure? Malcolm Macleod University of Edinburgh.
Summarising the evidence from animal models of neurological disease: Publication bias, poor internal validity, and (perhaps) some efficacy Malcolm Macleod.

Summarising the evidence from animal models of neurological disease: Publication bias, poor internal validity, and (perhaps) some efficacy Malcolm Macleod.
Modelling Stroke in the Laboratory - Separating Fact from Artefact The impact of sources of bias in animal models of neurological disease, and what we.

Modelling Stroke in the Laboratory - Separating Fact from Artefact The impact of sources of bias in animal models of neurological disease, and what we.
Summarising the evidence from animal models of neurological disease: Are there any babies in the bathwater? Malcolm Macleod University of Edinburgh.

Summarising the evidence from animal models of neurological disease: Are there any babies in the bathwater? Malcolm Macleod University of Edinburgh.
Preventing introduction of bias at the bench: from randomizing to experimental results meta-analysis Malcolm Macleod Centre for Clinical Brain Sciences,

Preventing introduction of bias at the bench: from randomizing to experimental results meta-analysis Malcolm Macleod Centre for Clinical Brain Sciences,
Summarising the evidence from animal models of neurological disease: Publication bias, poor internal validity, and (perhaps) some efficacy Malcolm Macleod.

Summarising the evidence from animal models of neurological disease: Publication bias, poor internal validity, and (perhaps) some efficacy Malcolm Macleod.

Similar presentations

Ads by Google