1. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 1 Two-sample comparisons Underlying principles Comparing population parameters: means, variances and medians Paired-sample tests Power analysis in two-sample comparisons

2. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 2 Concepts map

3. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 3 Two-sample comparisons Appropriate when there are two groups to compare (e.g. control and treatment). In principle, we can compare any sample statistic, e.g. group means, medians, variances, etc. Control Treatment Frequency s2Cs2C s2Ts2T

4. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 4 An example Two samples (1, 2) with mean values that differ by some amount . What is the probability p of observing this difference under H 0 that the two means are in fact equal? Frequency Sample 2 Sample 1

5. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 5 An example (cont’d) If H 0 is true, the expected distribution of the test statistic t is: -3-2 012 3 Probability (p) t Frequency Sample 2 Sample 1

6. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 6 An example (cont’d) For the two populations, suppose t = 2.01 What is the probability of getting a value at least this large under H 0 that the two means are in fact equal? Since p is small, it is unlikely that H 0 is true. Therefore, reject H 0. -3-2 0123 Probability t = 2.01 Frequency Sample 2 Sample 1

7. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 7 Two-sample comparisons: independent samples For independent-sample tests, there is no experimental correlation or “matching” between objects (observations) in the two groups. E.g. Weight at 6 months of a random sample of different piglets raised on two different diets. Diet

8. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 8 Two-sample comparisons: matched (paired) samples For paired sample tests, objects (observations) in one group are matched with objects in the other group. E.g. Weight at 6 months of 2 piglets, each from the same sow, raised on different diets. Diet Sows

9. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 9 Two-sample comparisons: control versus experiment Two plots of corn, one (control) with no treatment, the other (treatment) with nitrogen added Biological prediction: nitrogen increases crop yield H 0 :  T   C (one- tailed) Control Treatment Frequency Yield

10. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 10 Comparing means: the t-test Calculate difference between two means. H 0 (one-tailed): Calculate t and associated p with appropriate degrees of freedom Control Treatment Frequency Yield

11. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 11 What are degrees of freedom? The degrees of freedom for any estimated quantity (e.g. the mean, variance, etc.) is the total sample minus one… … because if you know the quantity (e.g. the mean) and the values of all n-1 observations, you know the value of the nth observation. l The degrees of freedom for any statistical model is the total sample size minus the number of parameters

12. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 12 Why do degrees of freedom matter? The distribution of the test statistic is different for different degrees of freedom. Therefore, depending on the degrees of freedom, the same difference in sample means will give different p values. -5-4-3-2012345 t 0.0 0.1 0.2 0.3 0.4 -5-4-3-2012345 0.0 0.1 0.2 0.3 0.4 Probability 8 df 1 df

13. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 13 Comparing means: the Mann-Whitney U test Want to compare yields of control and treatment, where each group has 4 replicate plots. Calculate rank sum (R C, R E ) for each group. H 0 : R C = R E Calculate U and associated p.

14. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 14 Comparing means: parametric (P) versus non-parametric (NP) tests *if assumptions are met

15. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 15 Independence of observations Lack of independence usually arises because observations are correlated in time or space E.g. measures of phosphorous concentrations upstream and downstream of a point source along a river. Upstream site Downstream site

16. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 16 Why do observations need to be independent? If observations are not independent, then the true degrees of freedom is less (sometimes much less) than the calculated degrees of freedom … … the distribution used to calculate p will be wrong … … and p will be smaller than it ought to be. -3-2 0123 Probability t calcuated df true df Calculated t

17. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 17 General procedure: N for each sample reasonably large (> 20) Evaluate independence assumption Test normality assumption. Test homoscedasticity assumption. If both samples are normal and have equal variances, use t-test (“pooled variances”). If both samples are normal, but have different variances, use Welch’s approximation (“separate variances”). If one or both samples are non-normal, try some transformation or use Mann-Whitney U.

18. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 18 N < 10 for each group Use Mann-Whitney U N < 10 for each group Use Mann-Whitney U General procedures N > 10 and N < 20 for each group Use both pooled or separate variances t-test and Mann-Whitney U… …and hope the results lead to the same inference (i.e. to reject or accept the null)! N < 10 for each group

19. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 19 Comparison of average size of sturgeon at The Pas and Cumberland House SYSTAT output from t- test:

20. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 20 Comparison of average size of sturgeon at The Pas and Cumberland House SYSTAT output from Mann-Whitney

21. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 21 Assessing normality Do normal probability plot. If plot appears linear to the eye, in general there is no need to go further. If you’re still concerned, run Lilliefors test.

22. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 22 The normal cumulative distribution Areas under the normal probability density function and the cumulative normal distribution function 0.2 0.4 0.6 0.8 1.0 0  2.28% 50.00% 68.27% p Normal probability density function Cumulative normal density function

23. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 23 Normal equivalent deviates Normal equivalent deviates Transformation of cumulative percentages into normal equivalent deviates (Z-scores) Normal equivalent deviates -1.64 -0.52 0.52 1.64 -1.28 0 1.28 0.05 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 0.95 1.00 Cumulative percent

24. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 24 Normal probability plots Examples of frequency distributions with their cumulative distributions plotted as normal probability plots. A: Normal; B: Equal Mixture of two distributions; C: Skewed to left; D: Skewed to right; E: Platykurtic; F: Leptokurtic. NED 3.72 -3.72 0 3.72 -3.72 0 3.72 -3.72 0 A B CD EF

25. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 25 Example (Lab 2): sturgeon size at The Pas and Cumberland House The normal probability plots for fklngth at The Pas and Cumberland House are:

26. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 26 Example: Sturgeon size (cont’d) Output from SYSTAT Lilliefors test: normality of fklngth at The Pas KOLMOGOROV-SMIRNOV ONE SAMPLE TEST USING STANDARD NORMAL VARIABLE N-OF-CASES MAXDIF LILLIEFORS PROBABILITY (2-TAIL) FKLNGTH 101.00000 0.07814 0.12662

27. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 27 Equality of variances (homoscedasticity) using the F-ratio test If variances are equal, then s 2 C = s 2 T. H 0 (F-ratio): This test is quite sensitive to non- normality. Control Treatment Frequency s2Cs2C s2Ts2T

28. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 28 Equality of variances (homoscedasticity) using Levene’s test If variances are equal, then s 2 C = s 2 T. H 0 (Levene’s): This test is less sensitive to non-normality. Control Treatment Frequency s2Cs2C s2Ts2T

29. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 29 Comparing medians: the median test Calculate median M for both samples combined. Classify each observation as being above or below M to create 2 X 2 table. Do  2 or G test of independence. Yield Frequency Control Experimental M

30. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 30 Paired-sample tests used when same object is measured under different treatments (e.g. change in rat weights before and after treatment with a drug)… … or when there is a correlation between observations in the two samples. Use paired t-statistic.

31. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 31 Paired- versus independent-sample t-tests When correlation present, paired t-test is much more powerful because standard deviation of average difference between pairs is usually much smaller than the standard error of the difference between the two means. If no correlation present, paired test weaker because N is number of pairs, not number of observations. s 2 b = 8.67, s 2 a = 21.58, s 2  W = 2.81

32. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 32 Paired versus independent-sample tests: changes in face width

33. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 33 Two-sample tests of the mean: minimum sample size Suppose we want to detect a difference between two sample means of at least . To test at the  significance level with 1 -  power, we can calculate the minimum sample size n min required to detect , given a pooled sample variance s p 2. Frequency Sample 1 Sample 2

34. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 34 Index of effect size Calculation t-Test on Means d Power analysis with G*Power

35. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 35 Comparison of average size of sturgeon at The Pas and Cumberland House SYSTAT output from t- test:

36. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 36 Index of effect size Calculation t-Test on Means d Power analysis with G*Power

37. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 37 Minimum sample size

38. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 38 Two-sample tests of the mean: minimal detectable difference What is the minimal detectable difference  min between two sample means that can be detected at the  significance level with 1 -  power, given an estimated pooled variance s p 2 ? Frequency Sample 1 Sample 2

39. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 39 Minimal detectable difference

40. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 40

41. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 41 An example: power of a two-sample test of the mean What is probability of detecting a true difference of 1.01 if  (2) =.01?

42. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 42 Calculating power

43. University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 23/10/2015 9:22 PM 43 Calculating power