1. Center for Biofilm Engineering Al Parker, PhD, Biostatistician Center for Biofilm Engineering Montana State University Statistics and Biofilms June 29, 2012

2. Standardized Biofilm Methods Laboratory Darla Goeres Al Parker Marty Hamilton Diane Walker Lindsey Lorenz Paul Sturman Kelli Buckingham- Meyer

3. What is statistical thinking?  Data  Experimental Design  Uncertainty and variability assessment

4. What is statistical thinking?  Data (pixel intensity in an image? log(cfu) from viable plate counts?)  Experimental Design - controls - randomization - replication (How many coupons? experiments? technicians? labs?)  Uncertainty and variability assessment

5. Why statistical thinking?  Anticipate criticism (design method and experiments accordingly)  Provide convincing results (establish statistical properties)  Increase efficiency (conduct the least number of experiments)  Improve communication

6. Why statistical thinking? Standardized Methods

7. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

8. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

9. A standard laboratory method is said to be relevant to a real-world scenario if, given the same inputs, the laboratory outcome is equivalent to the real-world outcome. Relevance

10. Elbow Prosthesis - in vivo study

11. Urinary catheter in vivo study

12. Urinary Catheter Biofilm

13. CV Catheter in vivo study

14. Biofilm in the Catheter Tip 1,000 X magnification Sheep (control)

15. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

16. A standard laboratory method is said to be reasonable if the method can be performed inexpensively using typical microbiological techniques and equipment. Reasonableness

17. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

18. Resemblance of Controls Independent runs of the method produce nearly the same control data, as indicated by a small standard deviation. Statistical tool: nested analysis of variance (ANOVA)

19. 86 mm x 128 mm plastic plate with 96 wells Lid has 96 pegs Resemblance Example: MBEC

20. 123456789101112 A100 50:NNGCSC B50 50:NNGCSC C25 50:NNGCSC D12.5 50:NNGC E6.25 50:NNGC F3.125 50:NNGC G1.563 50:NNGC H0.781 50:NNGC MBEC Challenge Plate disinfectant neutralizer test control

21. Resemblance Example: MBEC Mean LD= 5.55 Control Data: log 10 (cfu/mm 2 ) from viable plate counts rowcfu/mm 2 log(cfu/mm 2 ) A 5.15 x 10 5 5.71 B 9.01 x 10 5 5.95 C 6.00 x 10 5 5.78 D 3.00 x 10 5 5.48 E 3.86 x 10 5 5.59 F 2.14 x 10 5 5.33 G 8.58 x 10 4 4.93 H 4.29 x 10 5 5.63

22. ExpRow Control LD Mean LDSD 1A5.71 5.550.31 1B5.95 1C5.78 1D5.48 1E5.59 1F5.33 1G4.93 1H5.63 2A5.41 0.17 2B5.71 2C5.54 2D5.33 2E5.11 2F5.48 2G5.33 2H5.41 Resemblance Example: MBEC

23. Resemblance from experiment to experiment Mean LD = 5.48 S r = 0.26 the typical distance between a control well LD from an experiment and the true mean LD

24. Resemblance from experiment to experiment The variance S r 2 can be partitioned: 98% due to among experiment sources 2% due to within experiment sources

25. S n c m c 2 + Formula for the SE of the mean control LD, averaged over experiments S c = within-experiment variance of control LDs S E = among-experiment variance of control LDs n c = number of control replicates per experiment m = number of experiments 2 2 S m E 2 SE of mean control LD = CI for the true mean control LD = mean LD ± t m-1 x SE

26. 8 2 Formula for the SE of the mean control LD, averaged over experiments S c = 0.02 x (0.26) 2 = 0.00124 S E = 0.98 x (0.26) 2 = 0.06408 n c = 8 m = 2 2 2 2 SE of mean control LD = 0.00124 + 0.06408 = 0.1792 95% CI for the true mean control LD = 5.48 ± 12.7 x 0.1792 = (3.20, 7.76)

27. Resemblance from technician to technician Mean LD = 5.44 S r = 0.36 the typical distance between a control well LD and the true mean LD

28. The variance S r 2 can be partitioned: 0% due to technician sources 24% due to between experiment sources 76% due to within experiment sources Resemblance from technician to technician

29. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

30. Repeatability Independent runs of the method in the same laboratory produce nearly the same outcome, as indicated by a small repeatability standard deviation. Statistical tool: nested ANOVA

31. Repeatability Example Data: log reduction (LR) LR = mean(control LDs) – mean(disinfected LDs)

32. ExpRow Control LD Mean LDSD 1A5.71 5.550.31 1B5.95 1C5.78 1D5.48 1E5.59 1F5.33 1G4.93 1H5.63 2A5.41 0.17 2B5.71 2C5.54 2D5.33 2E5.11 2F5.48 2G5.33 2H5.41 Repeatability Example: MBEC 123456789101112 A 100 50:NNGCSC B 50 50:NNGCSC C 25 50:NNGCSC D 12.5 50:NNGC E 6.25 50:NNGC F 3.125 50:NNGC G 1.563 50:NNGC H 0.781 50:NNGC

33. Repeatability Example: MBEC Mean LR = 1.63 ExpRow Control LD Control Mean LDCol Disinfected 6.25% LD Disinfected Mean LDLR 1A5.71 5.554.511.04 1B5.9514.67 1C5.7824.41 1D5.4834.33 1E5.5944.59 1F5.3354.54 1G4.93 1H5.63 2A5.41 3.202.21 2B5.7114.78 2C5.5422.71 2D5.3333.48 2E5.1143.23 2F5.4851.82 2G5.33 2H5.41

34. Repeatability Example Mean LR = 1.63 S r = 0.83 the typical distance between a LR for an experiment and the true mean LR

35. S n c m c 2 + Formula for the SE of the mean LR, averaged over experiments S c = within-experiment variance of control LDs S d = within-experiment variance of disinfected LDs S E = among-experiment variance of LRs n c = number of control replicates per experiment n d = number of disinfected replicates per experiment m = number of experiments 2 2 2 S n d m d 2 + S m E 2 SE of mean LR =

36. Formula for the SE of the mean LR, averaged over experiments S c = within-experiment variance of control LDs S d = within-experiment variance of disinfected LDs S E = among-experiment variance of LRs n c = number of control replicates per experiment n d = number of disinfected replicates per experiment m = number of experiments 2 2 2 CI for the true mean LR = mean LR ± t m-1 x SE

37. Formula for the SE of the mean LR, averaged over experiments S c 2 = 0.00124 S d 2 = 0.47950 S E 2 = 0.59285 n c = 8, n d = 5, m = 2 SE of mean LR = 8 2 2 0.00124 + 0.59285 5 2 0.47950 + = 0.5868 95% CI for the true mean LR= 1.63 ± 12.7 x 0.5868 = 1.63 ± 7.46 = (0.00, 9.09)

38. How many coupons? experiments? n c m m 0.00124 + 0.59285 n d m 0.47950 + margin of error= t m-1 x no. control coupons (n c ): 235812 no. disinfected coupons (n d ): 235512 no. experiments (m) 28.207.807.46 7.16 32.272.152.06 1.97 41.451.381.32 1.27 60.960.910.87 0.84 100.650.620.59 0.57 1000.180.170.16

39. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

40. A method should be sensitive enough that it can detect important changes in parameters of interest. Statistical tool: regression and t-tests Responsiveness

41. disinfectant neutralizer test control Responsiveness Example: MBEC A: High Efficacy H: Low Efficacy 123456789101112 A100 50:NNGCSC B50 50:NNGCSC C25 50:NNGCSC D12.5 50:NNGC E6.25 50:NNGC F3.125 50:NNGC G1.563 50:NNGC H0.781 50:NNGC

42. Responsiveness Example: MBEC This response curve indicates responsiveness to decreasing efficacy between rows C, D, E and F

43. Responsiveness Example: MBEC Responsiveness can be quantified with a regression line: LR = 6.08 - 0.97row For each step in the decrease of disinfectant efficacy, the LR decreases on average by 0.97.

44. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

45. A standard laboratory method is said to be rugged if the outcome is unaffected by slight departures from the protocol. Ruggedness

46. Parameters in the protocol:  Sonication Power: 130, 250, 480 watts  Sonication Duration: 25, 30, 35 minutes  Treatment Temperature: 20, 22, 24 o C  Incubation Time: 16, 17, 18 hours Ruggedness Testing of the MBEC

47. Ruggedness Test Design Run Incubation Time Treatment Temperature Sonication Power Sonication Duration 117 hrs22°C250 watts30 218 hrs20°C130 watts25 316 hrs24°C480 watts35 418 hrs24°C480 watts25 518 hrs24°C130 watts35 618 hrs20°C480 watts35 716 hrs20°C480 watts25 817 hrs22°C250 watts30 916 hrs20°C130 watts35 1016 hrs24°C130 watts25

49. 123456789101112 A100 50:NNGCSC B50 50:NNGCSC C25 50:NNGCSC D12.5 50:NNGC E6.25 50:NNGC F3.125 50:NNGC G1.563 50:NNGC H0.781 50:NNGC MBEC Challenge Plate disinfectant neutralizer test control

50. Ruggedness Testing of the Controls

51. LD(controls) = 5.027 + 0.1111(IncubationTime – 17) - 0.0042(SonicationDuration -30) - 0.1178(TreatmentTemperature – 22) + 0.0004(SonicationPower – 250) + 0.3893(BiofilmGrowth – 5.87)  All terms are small and not of practical importance inside the range of values tested  None of the model terms were statistically significant  This model allows one to quantifiably predict how deviations from the protocol affect the experimental outcome

52. 123456789101112 A100 50:NNGCSC B50 50:NNGCSC C25 50:NNGCSC D12.5 50:NNGC E6.25 50:NNGC F3.125 50:NNGC G1.563 50:NNGC H0.781 50:NNGC MBEC Challenge Plate disinfectant neutralizer test control

53. Ruggedness Testing of the LRs

54. LR(H) = 0.2157 – 0.3738(IncubationTime – 17)* + 0.0015(SonicationDuration -30) – 0.1001(TreatmentTemperature – 22) + 0.0003(SonicationPower – 250)  Only IncubationTime was statistically significant*  Except for IncubationTime, the terms are small and not of practical importance inside the range of values tested

55. Ruggedness Testing of the LRs  Only IncubationTime was statistically significant*  Except for TreatmentTemperature, the terms are small and not of practical importance inside the range of values tested LR(A) = 5.7219 + 0.1254(IncubationTime – 17)* + 0.0015(SonicationDuration -30) – 0.2831(TreatmentTemperature – 22) + 0.0003(SonicationPower – 250)

56. Results of the ASTM ILS for the MBEC  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

57. Collaboration

59. ASTM Interlaboratory Study (ILS) Process  Register test method  Conduct ruggedness testing  Minimum of 6 participating labs  Technical contact Instructions Supplies Data template  Research report  Precision & Bias statement

60. ASTM ILS #25570  Eight labs  Three experimental test days at each lab  Three disinfectants tested/experiment day non-chlorine oxidizer phenol quaternary ammonium compound

61. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

62. Control Data

64. Untreated Control Variability Lab No Exp Mean LD Within plate % Among plate % Among exp day % Among lab % Repeatability SD Reproducibility SD 137.5040%34%25% 0.1369 237.5820%27%53% 0.4206 336.2739%12%49% 0.1696 437.9217%0%83% 0.2315 537.8064%0%36% 0.1624 637.728%7%85% 0.5301 738.1376%24%0% 0.1438 838.1651%0%49% 0.2706 All247.484%11%9%76%0.32520.6669

65. Attributes of a Standard Method: Seven R’s  Relevance  Reasonableness  Resemblance  Repeatability (intra-laboratory)  Responsiveness  Ruggedness  Reproducibility (inter-laboratory)

66. Independent runs of the method by different researchers in different laboratories produce nearly the same outcome (e.g. LR). This assessment requires a collaborative (multi- lab) study. Reproducibility

67. Treated Data: LR (Non-chlorine oxidizer)

68. Treated Data: LR (Phenol)

69. Treated Data: LR (Quat)

70. Oxidizer Results DisinfectantRowMean LR Within Among lab % Repeatability SD Reproducibility SD lab % Oxidizer A5.5075%25%1.05571.2205 B4.4196%4%1.49181.5196 C3.0392%8%1.60931.6771 D1.7285%15%1.56581.6986 E0.6050% 0.88441.2488 F-0.0834%66%0.37760.6453 G-0.19100%0%0.4687 H-0.18100%0%0.5223

71. Phenol Results DisinfectantRowMean LR Within Among lab % Repeatability SD Reproducibility SD lab % Phenol A5.64100%0%1.2578 B4.76100%0%1.2747 C2.5980%20%1.24671.3979 D1.1557%43%0.89841.1905 E0.3429%71%0.3260.6059 F-0.0252%48%0.25210.3509 G-0.1156%44%0.20150.2683 H-0.15100%0%0.3009

72. Quat Results DisinfectantRowMean LR Within Among lab % Repeatability SD Reproducibility SD lab % Quat A3.6436%64%0.90361.512 B2.2635%65%0.8621.4522 C1.3446%54%0.83721.2406 D0.9527%73%0.6061.1715 E0.5826%74%0.53021.0394 F0.1850% 0.39010.5501 G-0.0178%22%0.39440.4472 H-0.11100%0%0.3598

73. Repeatability at a glance …

74. Reproducibility at a glance …

75. ASTM Precision and Bias Statement Untreated Control Data Variance Assessment # of Labs # of Exps Mean LD a Sources of Variability Repeatability SD b Reproducibility SD b Within plate % Among plate % Among exp day % Amon g lab % 8247.484%11%9%76%0.32520.6669 12.0 Precision and Bias 12.1 Precision: 12.1.1 An interlaboratory study (ASTM ILS #650) of this test method was conducted at eight laboratories testing three disinfectants (non-chlorine oxidizer, phenol and quaternary ammonium compound) at 8 concentrations (depicted in Fig. 2). An ANOVA model was fit with random effects to determine the resemblance of the untreated control data and the repeatability and reproducibility of the treated data. 12.1.2 The reproducibility standard deviation was 0.67 for the mean log densities of the control biofilm bacteria for this protocol, based on averaging across eight wells on each plate. The sources of variability for the untreated control data are provided in Table 1. Table 1. Untreated control data variance assessment. 12.1.3 The repeatability (Fig. 5) and reproducibility (Fig. 6) of each disinfectant at each concentration is summarized. 12.1.4 For each of the three disinfectant types considered, the protocol was statistically significantly responsive to the increasing efficacy levels. The log reduction of the non-chlorine oxidizer increased by 0.87 for each increase in efficacy level. The log reduction of the phenol disinfectant increased by 0.87 for each increase in efficacy level. The log reduction of the quat increased by 0.5 for each increase in efficacy level. 12.2 Bias: 12.2.1 Randomization is used whenever possible to reduce the potential for systematic bias.

76. Summary  Even though biofilms are complicated, it is feasible to develop biofilm methods that meet the “Seven R” criteria  Good experiments use control data and randomization.  Invest effort in more experiments versus more replicates (coupons or wells) within an experiment.  Assess uncertainty by SEs and CIs.  For additional statistical resources for biofilm methods, check out: http://www.biofilm.montana.edu/category/documents/ksa-sm

77. Center for Biofilm Engineering A National Science Foundation Engineerin g Research Center established in 1990 www.biofilm.montana.edu Questions?