1. Center for Biofilm Engineering Al Parker, Biostatistician Standardized Biofilm Methods Research Team Montana State University The Importance of Statistical Design and Analysis in the Laboratory Feb, 2011

2. Standardized Biofilm Methods Laboratory Darla Goeres Al Parker Marty Hamilton Diane Walker Lindsey Lorenz Paul Sturman Kelli Buckingham- Meyer

3. What is statistical thinking?  Data  Experimental Design  Uncertainty and variability assessment

4. What is statistical thinking?  Data (pixel intensity in an image? log(cfu) from viable plate counts?)  Experimental Design - controls - randomization - replication (How many coupons? experiments? technicians? labs?)  Uncertainty and variability assessment

5. Why statistical thinking?  Anticipate criticism (design method and experiments accordingly)  Provide convincing results (establish statistical properties)  Increase efficiency (conduct the least number of experiments)  Improve communication

6. Why statistical thinking? Standardized Methods

7. Attributes of a standard method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Ruggedness  Responsiveness  Reproducibility (inter-laboratory)

8. Attributes of a standard method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Ruggedness  Responsiveness  Reproducibility (inter-laboratory)

9. Resemblance of Controls Independent repeats of the same experiment in the same laboratory produce nearly the same control data, as indicated by a small repeatability standard deviation. Statistical tool: nested analysis of variance (ANOVA)

10. 86 mm x 128 mm plastic plate with 96 wells Lid has 96 pegs Resemblance Example: MBEC

11. 123456789101112 A100 50:NNGCSC B50 50:NNGCSC C25 50:NNGCSC D12.5 50:NNGC E6.25 50:NNGC F3.125 50:NNGC G1.563 50:NNGC H0.781 50:NNGC MBEC Challenge Plate disinfectant neutralizer test control

12. Resemblance Example: MBEC Mean LD= 5.55 Control Data: log 10 (cfu/mm 2 ) from viable plate counts rowcfu/mm 2 log(cfu/mm 2 ) A 5.15 x 10 5 5.71 B 9.01 x 10 5 5.95 C 6.00 x 10 5 5.78 D 3.00 x 10 5 5.48 E 3.86 x 10 5 5.59 F 2.14 x 10 5 5.33 G 8.58 x 10 4 4.93 H 4.29 x 10 5 5.63

13. ExpRow Control LD Mean LDSD 1A5.71 5.550.31 1B5.95 1C5.78 1D5.48 1E5.59 1F5.33 1G4.93 1H5.63 2A5.41 0.17 2B5.71 2C5.54 2D5.33 2E5.11 2F5.48 2G5.33 2H5.41 Resemblance Example: MBEC

14. Resemblance from experiment to experiment Mean LD = 5.48 S r = 0.26 the typical distance between a control well LD from an experiment and the true mean LD

15. Resemblance from experiment to experiment The variance S r 2 can be partitioned: 2% due to between experiment sources 98% due to within experiment sources

16. S n c m c 2 + Formula for the SE of the mean control LD, averaged over experiments S c = within-experiment variance of control LDs S E = among-experiment variance of control LDs n c = number of control replicates per experiment m = number of experiments 2 2 S m E 2 SE of mean control LD = CI for the true mean control LD = mean LD ± t m-1 x SE

17. 8 2 Formula for the SE of the mean control LD, averaged over experiments S c = 0.98 x (0.26) 2 = 0.00124 S E = 0.02 x (0.26) 2 = 0.06408 n c = 8 m = 2 2 2 2 SE of mean control LD = 0.00124 + 0.06408 = 0.1792 95% CI for the true mean control LD = 5.48 ± 12.7 x 0.1792 = (3.20, 7.76)

18. Resemblance from technician to technician Mean LD = 5.44 S r = 0.36 the typical distance between a control well LD and the true mean LD

19. The variance S r 2 can be partitioned: 0% due to technician sources 24% due to between experiment sources 76% due to within experiment sources Resemblance from technician to technician

20. Repeatability Independent repeats of the same experiment in the same laboratory produce nearly the same data, as indicated by a small repeatability standard deviation. Statistical tool: nested ANOVA

21. Repeatability Example Data: log reduction (LR) LR = mean(control LDs) – mean(disinfected LDs)

22. ExpRow Control LD Mean LDSD 1A5.71 5.550.31 1B5.95 1C5.78 1D5.48 1E5.59 1F5.33 1G4.93 1H5.63 2A5.41 0.17 2B5.71 2C5.54 2D5.33 2E5.11 2F5.48 2G5.33 2H5.41 Repeatability Example: MBEC 123456789101112 A 100 50:NNGCSC B 50 50:NNGCSC C 25 50:NNGCSC D 12.5 50:NNGC E 6.25 50:NNGC F 3.125 50:NNGC G 1.563 50:NNGC H 0.781 50:NNGC

23. Repeatability Example: MBEC Mean LR = 1.63 ExpRow Control LD Control Mean LDCol Disinfected 6.25% LD Disinfected Mean LDLR 1A5.71 5.554.511.04 1B5.9514.67 1C5.7824.41 1D5.4834.33 1E5.5944.59 1F5.3354.54 1G4.93 1H5.63 2A5.41 3.202.21 2B5.7114.78 2C5.5422.71 2D5.3333.48 2E5.1143.23 2F5.4851.82 2G5.33 2H5.41

24. Repeatability Example Mean LR = 1.63 S r = 0.83 the typical distance between a LR for an experiment and the true mean LR

25. S n c m c 2 + Formula for the SE of the mean LR, averaged over experiments S c = within-experiment variance of control LDs S d = within-experiment variance of disinfected LDs S E = among-experiment variance of LRs n c = number of control replicates per experiment n d = number of disinfected replicates per experiment m = number of experiments 2 2 2 S n d m d 2 + S m E 2 SE of mean LR =

26. Formula for the SE of the mean LR, averaged over experiments S c = within-experiment variance of control LDs S d = within-experiment variance of disinfected LDs S E = among-experiment variance of LRs n c = number of control replicates per experiment n d = number of disinfected replicates per experiment m = number of experiments 2 2 2 CI for the true mean LR = mean LR ± t m-1 x SE

27. Formula for the SE of the mean LR, averaged over experiments S c 2 = 0.00124 S d 2 = 0.47950 S E 2 = 0.59285 n c = 8, n d = 5, m = 2 SE of mean LR = 8 2 2 0.00124 + 0.59285 5 2 0.47950 + = 0.5868 95% CI for the true mean LR= 1.63 ± 12.7 x 0.5868 = 1.63 ± 7.46 = (0.00, 9.09)

28. How many coupons? experiments? n c m m 0.00124 + 0.59285 n d m 0.47950 + margin of error= t m-1 x no. control coupons (n c ): 235812 no. disinfected coupons (n d ): 235512 no. experiments (m) 28.207.807.46 7.16 32.272.152.06 1.97 41.451.381.32 1.27 60.960.910.87 0.84 100.650.620.59 0.57 1000.180.170.16

29. A method should be sensitive enough that it can detect important changes in parameters of interest. Statistical tool: regression and t-tests Responsiveness

30. disinfectant neutralizer test control Responsiveness Example: MBEC A: High Efficacy H: Low Efficacy 123456789101112 A100 50:NNGCSC B50 50:NNGCSC C25 50:NNGCSC D12.5 50:NNGC E6.25 50:NNGC F3.125 50:NNGC G1.563 50:NNGC H0.781 50:NNGC

31. Responsiveness Example: MBEC This response curve indicates responsiveness to decreasing efficacy between rows C, D, E and F

32. Responsiveness Example: MBEC Responsiveness can be quantified with a regression line: LR = 6.08 - 0.97row For each step in the decrease of disinfectant efficacy, the LR decreases on average by 0.97.

33. Summary  Even though biofilms are complicated, it is feasible to develop biofilm methods that meet the “Seven R” criteria.  Good experiments use control data!  Assess uncertainty by SEs and CIs.  When designing experiments, invest effort in more experiments versus more replicates (coupons or wells) within an experiment.

34. Any questions?