1. CAMARADES: Bringing evidence to translational medicine Optimizing the Predictive Value of Pre-Clinical Research Session 3: Reviewer Perspective Malcolm Macleod Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies and Centre for Clinical Brain Sciences, University of Edinburgh

2. CAMARADES: Bringing evidence to translational medicine Quality is important EAE PD AD

3. CAMARADES: Bringing evidence to translational medicine Prevalence of reporting of some measures to improve validity Randomisation Blinded Outcome Assessment Sample Size calculation Stroke36%29%3% MND31%20%<1% AD15%25%0% PD12%15%0% EAE8%15%<1% Glioma14%0% Pain14%25%0%

4. CAMARADES: Bringing evidence to translational medicine Quality by Journal 4584 full publications curated on CAMARADES Reporting the efficacy of an intervention in an animal disease model Journals contributing more than 100 publications –Brain Research –Experimental Neurology –JCBFM* –Journal of Immunology –Journal of Neuroimmunology –Journal of Neuroscience –Neuroscience –PNAS –Stroke

5. CAMARADES: Bringing evidence to translational medicine Quality by Journal

6. CAMARADES: Bringing evidence to translational medicine Does high Impact Factor reflect high quality research? 563 publications in focal cerebral ischaemia Weak association between Impact Factor and quality (adjusted r 2 = 0.06) Weaker association between number of citations received by that publication and quality (adjusted r 2 = 0.004)

7. CAMARADES: Bringing evidence to translational medicine Change over time EAE: some improvement over time: 26 years per point increment in quality AD: some improvement over time: 24 years per point improvement in study quality Study quality of in vivo studies selected from random sample of 1000 publications from PubMed Randomisation Blinded outcome assessment Blinded conduct of experiment Concealment of allocation sequence Sample size calculation Conflict of Interest statement

8. CAMARADES: Bringing evidence to translational medicine

9. Publication bias n expts Estimated unpublished Reported efficacy Corrected efficacy Stroke – infarct volume135921431.3%23.8% EAE - neurobehaviour189250533.1%15.0% EAE – inflammation8181438.2%37.5% EAE – demyelination2907445.1%30.5% EAE – axon loss1704654.8%41.7% AD – Water Maze80150.688 sd0.498 sd AD – plaque burden6321540.999 sd0.610 sd - 32%20%

10. CAMARADES: Bringing evidence to translational medicine Dimensions of assessment Internal validity Efficacy Generalisability 1234 1332 22221 31232 41114 1234 13 232 31442 4641 Validity Efficacy 1234 1111 2221 33233 45222 Exemplar heat map of 30 experiments testing an intervention in EAE

11. CAMARADES: Bringing evidence to translational medicine Evidence based translational medicine Experimental Studies Systematic review and meta-analysis how powerful is the treatment? what is the quality of evidence? what is the range of evidence? is there evidence of a publication bias? What are the conditions of maximum efficacy? Multi Centre Animal Studies confirm efficacy robust and monitored conduct of experiments transparent analysis and reporting deliberate heterogeneity Clinical trial