1. 1 The design of animal experiments Michael FW Festing c/o Understanding Animal Research, 25 Shaftsbury Av. London, UK. michaelfesting@aol.com michaelfesting@aol.com

2. 2 ) Replacement 5 e.g. in-vitro methods, less sentient animals ) Refinement 5 e.g. anaesthesia and analgesia, environmental enrichment ) Reduction 5 Research strategy 5 Controlling variability 5 Experimental design and statistics Principles of Humane Experimental Technique (Russell and Burch 1959)

3. 3 A well designed experiment Absence of bias Experimental unit, randomisation, blinding High power Low noise ( uniform material, blocking, covariance) High signal (sensitive subjects, high dose) Large sample size Wide range of applicability Replicate over other factors (e.g. sex, strain): factorial designs Simplicity Amenable to a statistical analysis

4. 4 The animal as the experimental unit Animals individually treated. May be individually housed or grouped N=8 n=4

5. 5 A cage as the Experimental Unit. Treatment in water or diet. N=4 n=2 Treated Control

6. 6 An animal for a period of time: repeated measures or crossover design Animal 1 2 3 Treatment 1 Treatment 2 N444N444 N=12 n= 6

7. 7 Teratology: mother treated, young measured Mother is the experimental unit. N=2 n=1

8. 8 Failure to identify the experimental unit correctly in a 2(strains) x 3(treatments) x 6(times) factorial design ELD group Single cage of 8 mice killed at each time point (288 mice in total)

9. 9 Experimental units must be randomised to treatments Physical: numbers on cards. Shuffle and take one Tables of random numbers in most text books Use computer. e.g. EXCEL or a statistical package such as MINITAB

10. 10 Randomisation Original Randomised 12 13 12 212 3 323 31 NB Randomisation should include housing and order in which observations are made

11. 11 Failure to randomise and/or blind leads to more “positive” results Blind/not blind odds ratio 3.4 (95% CI 1.7-6.9) Random/not random odds ratio 3.2 (95% CI 1.3-7.7) Blind Random/ odds ratio 5.2 (95% CI 2.0-13.5) not blind random 290 animal studies scored for blinding, randomisation and positive/negative outcome, as defined by authors Babasta et al 2003 Acad. emerg. med. 10:684-687

12. 12 Some factors (e.g. strain, sex) can not be randomised so special care is needed to ensure comparability Outbred TO (8-12 weeks commercial) Inbred CBA (12-16 weeks Home bred) Six cages of 7-9 mice of each strain: error bars are SEMs "CBA mice showed greater variability in body weights than TO mice..."

13. 13 A well designed experiment Absence of bias Experimental unit, randomisation, blinding High power Low noise ( uniform material, blocking, covariance) High signal (sensitive subjects, high dose) Large sample size Wide range of applicability Replicate over other factors (e.g. sex, strain): factorial designs Simplicity Amenable to a statistical analysis

14. 14 High power: (good chance of detecting the effect of a treatment, if there is one) High Signal/Noise ratio = High Standardized effect size = High  |  1 -  2 |/  = High  Difference between means)/SD Student’s t =( X 1 - X 2 ) / Sqrt (2S 2 /n)

15. 15 Power Analysis for sample size and effects of variation A mathematical relationship between six variables Needs subjective estimate of effect size to be detected (signal) Has to be done separately for each character Not easy to apply to complex designs Essential for expensive, simple, large experiments (clinical trials) Useful for exploring effect of variability A second method “The Resource Equation” is described later

16. 16 Power analysis: the variables Sample size Signal a) Effect size of scientific interest or b) actual response Chance of a false positive result. Significance level (0.05) Sidedness of statistical test (usually 2-sided) Power of the Experiment (80-90%?) Noise Variability of the experimental material

17. 17 Group size and Signal/noise ratio Assuming 2-sample, 2 sided t-test and 5% significance level Signal/noise ratio Power Neutral Bad Good

18. 18 Comparison of two anaesthetics for dogs under clinical conditions (Vet. Anaesthes. Analges.) Unsexed healthy clinic dogs, Weight 3.8 to 42.6 kg. Systolic BP 141 (SD 36) mm Hg Assume: a 20 mmHg difference between anaesthetics is of clinical importance, a significance level of  =0.05 a power=90% a 2-sided t-test Signal/Noise ratio 20/36 = 0.56 Required sample size 68/group

19. 19 Power and sample size calculations using nQuery Advisor

20. 20 A second paper described: Male Beagles weight 17-23 kg mean BP 108 (SD 9) mm Hg. Want to detect 20mm difference between groups (as before) With the same assumptions as previous slide: Signal/noise ratio = 20/9 = 2.22 Required sample size 6/group

21. 21 Summary for two sources of dogs: aim is to be able to detect a 20mmHg change in blood pressure Type of dog SDev Signal/noise Sample %Power (n=8) size/gp(1) (2) Random dogs 36 0.56 68 18 Male beagles 9 2.22 6 98 (1)Sample size: 90% power (2)Power, Sample size 8/group Assumes  =5%, 2-sided t-test and effect size 20mmHg The scientific dilemma: With small sample sizes we can not detect an important effect in genetically heterogeneous animals. We can detect the effect in genetically homogeneous animals, but are they representative?

22. 22 Variation in kidney weight in 58 groups of rats Gartner,K. (1990), Laboratory Animals, 24:71-77.

23. 23 Required sample sizes FactorTypeStd.DevSignal/ noise* Sample size Power** GeneticsF1 hybrid13.50.743080 F2 hybrid18.40.545553 Outbred20.10.496746 DiseaseMycoplasma free 18.60.545553 With Mycoplasma 43.30.2329814 *signal is 10 units, two sided t-test,  =0.05, power = 80% ** Assuming fixed sample size of 30/group

24. 24 The randomised block design: another method of controlling noise BCA ACB BAC ACB BCA B1 B2 B3 B4 B5 Treaments A, B & C Randomisation is within-block Can be multiple differences between blocks Heterogeneous age/weight Different shelves/rooms Natural structure (litters) Split experiment in time

25. 25 A randomised block experiment 365 398 421 423 432 459 308 320 329 Treatment effect p=0.023 (2-way ANOVA)

26. 26 Analysis of apoptosis data Analysis of Variance for Score Source DF SS MS F P Block 2 21764.2 10882.1 114.82 0.000 Treatmen 2 2129.6 1064.8 11.23 0.023 Error 4 379.1 94.8 Total 8 24272.9

27. 27

28. 28 Another method of determining sample size: The Resource Equation Depends on the law of diminishing returns Simple. No subjective parameters Useful for complex designs and/or multiple outcomes (characters) Does not require estimate of Standard Deviation Crude compared with Power Analysis E= (Total number of animals)-(number of groups) 10<E<20 (but give some tolerance)

29. 29 But if experimental subjects are cheap (e.g. multi-well plates, E can be much higher

30. 30 A well designed experiment Absence of bias Experimental unit, randomisation, blinding High power Low noise ( uniform material, blocking, covariance) High signal (sensitive subjects, high dose) Large sample size Wide range of applicability Replicate over other factors to (e.g. sex, strain) to increase generality: factorial designs Simplicity Amenable to a statistical analysis

31. 31 Factorial designs Single factor design Treated Control E=16-2 = 14 One variable at a time (OVAT) Treated Control E=16-2 = 14 Factorial design Treated Control E=16-4 = 12

32. 32 Factorial designs (By using a factorial design)”.... an experimental investigation, at the same time as it is made more comprehensive, may also be made more efficient if by more efficient we mean that more knowledge and a higher degree of precision are obtainable by the same number of observations.” R.A. Fisher, 1960

33. 33 A 4x2 factorial design Analysed with Student’s t-test: This is not appropriate because: 1.Each test is based on too few animals (n=3-4), so lacks power 2.It does not indicate whether there are strain differences in protein thiol status 3.It does not indicate whether dose/response differs between strains 4.A two-way design should be analysed using a 2-way ANOVA

34. 34 Incorrect statistical analysis leading to excessive numbers of animals 8 mice per group 8groups = 64 mice. E= 64-8 =56 Alternative 3 mice per group: 8 groups E=24-8 = 16 Saving:40 mice Formal test of interaction One experiment or 4 separate experiments?

35. 35 2 (strains) x 4 (Animal units) factorial

36. 36 Effect of chloramphenicol (2000mg/kg) on RBC count StrainControl Treated C3H7.857.81 8.777.21 8.486.96 8.227.10 CD-19.019.18 7.768.31 8.428.47 8.838.67 Tests: Use a two-way ANOVA with interaction 1. Do the treatment means averaged across strains differ? 2. Do the strains differ, averaged across treatments 3. Do the two strains respond to the same extent? Should not be analysed using two t-tests 1. Each test lacks power due to small sample size 2. Will not give a test of whether strains differ in response

37. 37 A 2x2 factorial design with interaction Source DF SS MS F P strain 1 2.4414 2.4414 13.13 0.003 Treatment 1 0.8236 0.8236 4.43 0.057 strain*treat. 1 1.4702 1.4702 7.91 0.016 Error 12 2.2308 0.1859 Total 15 6.9659 C3H CD-1 Pooled variance

38. 38 Use of several inbred strains to reduce noise, increase signal and explore generality 5001000150020002500 CD-18 8 8 8 8 8 CBA2 2 2 2 2 C3H2 2 2 2 2 BALB/c 2 2 2 2 2 C57BL2 2 2 2 2 2 2 2 2 Inbred 0 Outbred Dose of chloramphenicol (mg/kg) Festing et al (2001) Fd. Chem.Tox. 39:375 Effect of chloramphenicol on mouse haematology

39. 39 WBC Strain ControlTreated CBA1.900.40 CBA2.600.20 C3H2.100.40 C3H2.200.40 BALB/c1.601.30 BALB/c0.501.40 C57BL2.300.80 C57BL2.201.10 CD-13.001.90 CD-11.701.90 CD-11.503.50 CD-12.001.20 CD-13.802.30 CD-10.901.00 CD-12.601.30 CD-12.301.60 Example of a factorial compared with a single factor design Four inbred strains One outbred stock

40. 40 Signal Noise Strain N 0 2500 (Difference) (SD) Signal/noise p CBA4 2.25 0.30 1.95 0.34 5.73 C3H4 2.15 0.40 1.85 0.34 5.44 BALB/c4 1.05 1.35 (-0.30) 0.34 (-0.88) C57BL4 2.25 0.95 1.30 0.34 3.82 Mean 16 1.93 1.20 0.73 0.34 2.15 <0.001 Dose * strain <0.001 WBC counts following chloramphenicol at 2500mg/kg Signal Noise Strain N 0 2500 (Difference) (SD) Signal/noise p CD-1 16 2.23 1.83 0.40 0.86 0.47 0.38 White blood cell counts

41. 41 Genetics is important: Twenty two Nobel Prizes since 1960 for work depending on inbred strains Cancer mmTV Transmissable encephalopathacies/prions Pruisner Retroviruses, Oncogenes & growth factors Cohen, Levi-montalcini, Varmus, Bishop, Baltimore, Temin Humoral immunity/antibodies T-cell receptor Tonegawa, Jerne Cell mediated immunity Immunological tolerance H2 restriction, immune responses Medawar, Burnet, Doherty, Zinkanagel Benacerraf (G.pigs) Genetics Snell C.C. Little, DBA, 1909 Inbred Strains and derivatives Jackson Laboratory monoclonal antibodies BALB/c mice Kohler and Millstein Smell Axel & Buck ES cells & “knockouts” Evans, Capecchi, Smithies

42. 42 18 th Annual Short Course on Experimental Models of Human Cancer August 21-30, 2009 Bar Harbor, ME courses.jax.org

43. 43 Conclusions Five requirements for a good design Unbiased (randomisation, blinding) Powerful (signal/noise ratio: control variability) Wide range of applicability (factorial designs, common but frequently analysed incorrectly) Simple Amenable to statistical analysis Mistakes in design and analysis are common Better training in experimental design would improve the quality of research, save money, time and animals

44. 44

45. 45