1. Emily Sena, Gillian Currie, Hanna Vesterinen, Kieren Egan, Nicki Sherratt, Cristina Fonseca, Zsannet Bahor, Jing Liao, Theo Hirst, Kim Wever, Hugo Pedder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru, Stelios Serghiou, Aaron McLean, Catherine Dick, Tracey Woodruff, Patrice Sutton, Andrew Thomson, Aparna Polturu, Sarah MaCann, Gillian Mead, Joanna Wardlaw, Rustam Salman, Joseph Frantzias, Robin Grant, Paul Brennan, Ian Whittle, Andrew Rice, Rosie Moreland, Nathalie Percie du Sert, Paul Garner, Lauralyn McIntyre, Gregers Wegener, Lindsay Thomson, David Howells, Ana Antonic, Tori O’Collins, Uli Dirnagl, H Bart van der Worp, Philip Bath, Mharie McRae, Stuart Allan, Ian Marshall, Xenios Mildonis, Konstantinos Tsilidis, Orestis Panagiotou, John Ioannidis, Peter Batchelor, David Howells, Sanne Jansen of Lorkeers, Geoff Donnan, Peter Sandercock, Emily Sena, Gillian Currie, Hanna Vesterinen, Kieren Egan, Nicki Sherratt, Cristina Fonseca, Zsannet Bahor, Jing Liao, Theo Hirst, Kim Wever, Hugo Pedder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru, Stelios Serghiou, Aaron McLean, Catherine Dick, Tracey Woodruff, Patrice Sutton, Andrew Thomson, Aparna Polturu, Sarah MaCann, Gillian Mead, Joanna Wardlaw, Rustam Salman, Joseph Frantzias, Robin Grant, Paul Brennan, Ian Whittle, Andrew Rice, Rosie Moreland, Nathalie Percie du Sert, Paul Garner, Lauralyn McIntyre, Gregers Wegener, Lindsay Thomson, David Howells, Ana Antonic, Tori O’Collins, Uli Dirnagl, H Bart van der Worp, Philip Bath, Mharie McRae, Stuart Allan, Ian Marshall, Xenios Mildonis, Konstantinos Tsilidis, Orestis Panagiotou, John Ioannidis, Peter Batchelor, David Howells, Sanne Jansen of Lorkeers, Geoff Donnan, Peter Sandercock, Emily Sena, Gillian Currie, Hanna Vesterinen, Kieren Egan, Nicki Sherratt, Cristina Fonseca, Zsannet Bahor, Jing Liao, Theo Hirst, Kim Wever, Hugo Pedder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru, Stelios Serghiou, Aaron McLean, Catherine Dick, Tracey Woodruff, Patrice Sutton, Andrew Thomson, Aparna Polturu, Sarah MaCann, Gillian Mead, Joanna Wardlaw, Rustam Salman, Joseph Frantzias, Robin Grant, Paul Brennan, Ian Whittle, Andrew Rice, Rosie Moreland, Nathalie Percie du Sert, Paul Garner, Lauralyn McIntyre, Gregers Wegener, Lindsay Thomson, David Howells, Ana Antonic, Tori O’Collins, Uli Dirnagl, H Bart van der Worp, Philip Bath, Mharie McRae, Stuart Allan, Ian Marshall, Xenios Mildonis, Konstantinos Tsilidis, Orestis Panagiotou, John Ioannidis, Peter Batchelor, David Howells, Sanne Jansen of Lorkeers, Geoff Donnan, Peter Sandercock, Emily Sena, Gillian Currie, Hanna Vesterinen, Kieren Egan, Nicki Sherratt, Cristina Fonseca, Zsannet Bahor, Jing Liao, Theo Hirst, Kim Wever, Hugo Pedder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru, Stelios Serghiou, Aaron McLean, Catherine Dick, Tracey Woodruff, Patrice Sutton, Andrew Thomson, Aparna Polturu, Sarah MaCann, Gillian Mead, Joanna Wardlaw, Rustam Salman, Joseph Frantzias, Robin Grant, Paul Brennan, Ian Whittle, Andrew Rice, Rosie Moreland, Nathalie Percie du Sert, Paul Garner, Lauralyn McIntyre, Gregers Wegener, Lindsay Thomson, David Howells, Ana Antonic, Tori O’Collins, Uli Dirnagl, H Bart van der Worp, Philip Bath, Mharie McRae, Stuart Allan, Ian Marshall, Xenios Mildonis, Konstantinos Tsilidis, Orestis Panagiotou, John Ioannidis, Peter Batchelor, David Howells, Sanne Jansen of Lorkeers, Geoff Donnan, Peter Sandercock, Emily Sena, Gillian Currie, Hanna Vesterinen, Kieren Egan, Nicki Sherratt, Cristina Fonseca, Zsannet Bahor, Jing Liao, Theo Hirst, Kim Wever, Hugo Pedder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru, Stelios Serghiou, Aaron McLean, Catherine Dick, Tracey Woodruff, Patrice Sutton, Andrew Thomson, Aparna Polturu, Sarah MaCann, Gillian Mead, Joanna Wardlaw, Rustam Salman, Joseph Frantzias, Robin Grant, Paul Brennan, Ian Whittle, Andrew Rice, Rosie Moreland, Nathalie Percie du Sert, Paul Garner, Lauralyn McIntyre, Gregers Wegener, Lindsay Thomson, David Howells, Ana Antonic, Tori O’Collins, Uli Dirnagl, H Bart van der Worp, Philip Bath, Mharie McRae, Stuart Allan, Ian Marshall, Xenios Mildonis, Konstantinos Tsilidis, Orestis Panagiotou, John Ioannidis, Peter Batchelor, David Howells, Sanne Jansen of Lorkeers, Geoff Donnan, Peter Sandercock, CAMARADES: Bringing evidence to translational medicine Animal models for human disease How can we improve translation of effects to humans? Malcolm Macleod Professor of Neurology and Translational Neurosciences University of Edinburgh and Honorary Consultant Neurologist, NHS Forth Valley Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies

2. CAMARADES: Bringing evidence to translational medicine Disclosures Member of UK Home Office Animals in Science Committee Member, Commission for Human Medicines, UK MHRA Have sought and will seek funding for work in this area

3. CAMARADES: Bringing evidence to translational medicine I am not in the office at the moment. Send any work to be translated.

4. CAMARADES: Bringing evidence to translational medicine Winner of the 2012 Ignoble Prize for Neuroscience

5. CAMARADES: Bringing evidence to translational medicine Web based analysis tools Deighton et al 2010 Figure 2: Bootstrap of 1000 sets of 13 random proteins. A frequency distribution of the IPA scores from 1000 randomly generated sets of 13 proteins. The median IPA score is 16. Only 16 of the 1000 random sets have a score of 30 or greater.

6. CAMARADES: Bringing evidence to translational medicine 10 -120 M10 -60 M

7. CAMARADES: Bringing evidence to translational medicine Why preclinical trials fail to translate Animal studies mislead if their conclusions are confounded by bias because of how the experiment were done of how the data were analysed incomplete data are available the wrong sample size the wrong model

8. CAMARADES: Bringing evidence to translational medicine How the experiment was done … GroupDay 1 Day 2 Day 3 Day 4 Day 5 “Maze bright” 1.331.602.602.833.26 “Maze dull” 0.721.102.231.83 Δ+0.60+0.50+0.37+1.00+1.43 Rosenthal and Fode (1963), Behav Sci 8, 183-9 12 graduate psychology students 5 day experiment: rats in T maze with dark arm alternating at random, and the dark arm always reinforced 2 groups – “Maze Bright” and “Maze dull”

9. CAMARADES: Bringing evidence to translational medicine Evidence from various neuroscience domains … Multiple SclerosisParkinson´s disease Alzheimer´s diseaseStroke

10. CAMARADES: Bringing evidence to translational medicine Infarct Volume –11 publications, 29 experiments, 408 animals –Improved outcome by 44% (35-53%) Macleod et al, 2008 Efficacy  RandomisationBlinded assessment of outcome Blinded conduct of experiment Risk of bias in animal studies

11. CAMARADES: Bringing evidence to translational medicine The scale of the problem CAMARADES 2671

12. CAMARADES: Bringing evidence to translational medicine The scale of the problem PubMED 2000

13. CAMARADES: Bringing evidence to translational medicine 1173 publications using non human animals, published in 2009 or 2010, from 5 leading UK universities “an outstanding contribution to the internationally excellent position of the UK in biomedical science and clinical/translational research.” “impressed by the strength within the basic neurosciences that were returned …particular in the areas of behavioural, cellular and molecular neuroscience” The scale of the problem RAE 1173 Rand BlindI/ESSC

14. CAMARADES: Bringing evidence to translational medicine Reporting of randomisation across 3 datasets, 2009-10

15. CAMARADES: Bringing evidence to translational medicine Reporting of risk bias items by decile of journal impact factor

16. CAMARADES: Bringing evidence to translational medicine Are multiply positive studies plausible? Assume a publication describes 6 experiments Assume the mechanistic hypothesis is correct Assume individual experiments, using different cohorts, are powered at 50% Then – p(at least 1 outcome positive) = 98% – p(at least 2 outcomes positive)= 89% – p(3 outcomes positive)= 66% – … – p(6 outcomes positive) = 2%

17. CAMARADES: Bringing evidence to translational medicine Probability of at least n positive studies 0123456 6 studies100%98%89%66%35%12%3% 0123456 6 studies100%98%89%66%35%12%3% 12 studies100% 98%93%81%61%

18. CAMARADES: Bringing evidence to translational medicine Publication bias n expts Estimated unpublished Reported efficacy Corrected efficacy Stroke – infarct volume135921431.3%23.8% EAE - neurobehaviour189250533.1%15.0% EAE – inflammation8181438.2%37.5% EAE – demyelination2907445.1%30.5% EAE – axon loss1704654.8%41.7% AD – Water Maze80150.688 sd0.498 sd AD – plaque burden6321540.999 sd0.610 sd - 32%20%

19. CAMARADES: Bringing evidence to translational medicine Small group sizes and publication bias conspire together Simulation: 1000 studies Complete publication bias (anything p>0.05 unpublished) True effect size 10, SD 10 Number of animals per group 5101520 % of studies published30%54%76%86%

20. CAMARADES: Bringing evidence to translational medicine Validation of animal models Clot busting treatment for stroke

21. CAMARADES: Bringing evidence to translational medicine 43.5% protection (40.1-47.0) Hypothermia: How powerful is the treatment in animals? 101 publications 222 experiments 3256 animals

22. CAMARADES: Bringing evidence to translational medicine What is the quality of that evidence?

23. CAMARADES: Bringing evidence to translational medicine duration of ischaemia favours hypothermiafavours control What is the range of evidence? favours hypothermiafavours control presence of hypertension delay to treatment depth of hypothermia

24. CAMARADES: Bringing evidence to translational medicine EuroHYP-1 Knowledge translation table Criterion Animal dataEuroHYP-1 How powerful is the treatment? >40% improvement in outcome Powered to detect 7% improvement in outcome What is the quality of evidence? Efficacy maintained in high quality studies Randomised, blinded outcome assessment, intensely monitored Is there evidence of a publication bias? Yes, but >35% improvement in adjusted outcome Registered What is the range of evidence? Good: sex, duration, delay to treatment, intensity, hypertension, reperfusion Patients >18yo with acute ischaemic stroke, NIHSS 6 to 20, treated within 6hrs What are the conditions of maximum efficacy? Temperature dependent: otherwise robust across dimensions Target 34-35°C

25. CAMARADES: Bringing evidence to translational medicine How can we increase the chances of translational success? Systematically review the available data Conduct further in vivo experiments if indicated Design your clinical trial accordingly Develop tools to allow rapid, living systematic reviews

26. CAMARADES: Bringing evidence to translational medicine Improvement strategies Journals –ARRIVE reporting guidelines –LANDIS transparency guidelines –NPG publication policy –Audits and RCTs to improve uptake of these Funders –Emphasising rigour in grant award Institutions –Audit of performance –CPD opportunities for scientists

27. CAMARADES: Bringing evidence to translational medicine Improvement strategies Journals –Pre-registration of studies (Cortex) –Link to study protocols (eg Open Science) Funders –Pool published lay abstracts at time of award –Support development of standard format for study protocols Scientists –Publish protocols eg on Dryad, Figshare so DOI

28. CAMARADES: Bringing evidence to translational medicine Improvement strategies Journals –Are you the Sunday Sport, the Daily Mail or a journal of record? Funders –Withhold 10% of grant pending publication –Support new publication models Institutions –Encourage rapid publication anywhere, not vanity publishing in journals of the highest “impact” –On appointment panels, look at the work, not where it was published

29. CAMARADES: Bringing evidence to translational medicine Improvement strategies Funders –Require real power calculations –Provide expertise in experimental design (NC3Rs) Regulators –End the tyranny of “Reduction” Institutions –Provide in house expertise in statistical design Scientists –Cut your cloth according to your coat

30. CAMARADES: Bringing evidence to translational medicine Improvement strategies Scientists –Develop better validation of animal models Targeted experiments (ALZ.org) Systematic review of hundreds of experiments comparing models, outcome measures

31. CAMARADES: Bringing evidence to translational medicine If you are planning a systematic review or meta- analysis of animal data, CAMARADES are here to help: malcolm.macleod@ed.ac.uk