1. Lecture Presentation Slides SEVENTH EDITION STATISTICS Moore / McCabe / Craig Introduction to the Practice of Chapter 12 One-Way Analysis of Variance

2. 2 Chapter 12 One-Way Analysis of Variance 12.1 Inference for One-Way Analysis of Variance 12.2 Comparing the Means

3. 12.1 Inference for One-Way Analysis of Variance  The Idea of ANOVA  Comparing Several Means  The Problem of Multiple Comparisons  The ANOVA F Test 3

4. 4 Introduction The two sample t procedures of Chapter 7 compared the means of two populations or the mean responses to two treatments in an experiment. In this chapter we’ll compare any number of means using Analysis of Variance. Note: We are comparing means even though the procedure is Analysis of Variance.

5. The Idea of ANOVA When comparing different populations or treatments, the data are subject to sampling variability. We can pose the question for inference in terms of the mean response. Analysis of variance (ANOVA) is the technique used to compare several means. One-way ANOVA is used for situations in which there is only one way to classify the populations of interest. Two-way ANOVA is used to analyze the effect of two factors. 5

6. 6 The details of ANOVA are a bit daunting. The main idea is that when we ask if a set of means gives evidence for differences among the population means, what matters is not how far apart the sample means are, but how far apart they are relative to the variability of individual observations. The Analysis of Variance Idea Analysis of variance compares the variation due to specific sources with the variation among individuals who should be similar. In particular, ANOVA tests whether several populations have the same mean by comparing how far apart the sample means are with how much variation there is within the sample. The Analysis of Variance Idea Analysis of variance compares the variation due to specific sources with the variation among individuals who should be similar. In particular, ANOVA tests whether several populations have the same mean by comparing how far apart the sample means are with how much variation there is within the sample. The Idea of ANOVA

7. 7 The sample means for the three samples are the same for each set. The variation among sample means for (a) is identical to (b). The variation among the individuals within the three samples is much less for (b).  CONCLUSION: the samples in (b) contain a larger amount of variation among the sample means relative to the amount of variation within the samples, so ANOVA will find more significant differences among the means in (b) –assuming equal sample sizes here for (a) and (b). –Note: larger samples will find more significant differences. The Idea of ANOVA

8. 8 Comparing Several Means Do SUVs and trucks have lower gas mileage than midsize cars? Response variable: gas mileage (mpg) Groups: vehicle classification 31 midsize cars 31 SUVs 14 standard-size pickup trucks Data from the Environmental Protection Agency’s Model Year 2003 Fuel Economy Guide, www.fueleconomy.gov.

9. 9 Means: Midsize:27.903 SUV:22.677 Pickup:21.286 Mean gas mileage for SUVs and pickups appears less than for midsize cars. Are these differences statistically significant? Comparing Several Means

10. 10 Null hypothesis: The true means (for gas mileage) are the same for all groups (the three vehicle classifications). We could look at separate t tests to compare each pair of means to see if they are different: 27.903 vs. 22.677, 27.903 vs. 21.286, & 22.677 vs. 21.286 H 0 : μ 1 = μ 2 H 0 : μ 1 = μ 3 H 0 : μ 2 = μ 3 However, this gives rise to the problem of multiple comparisons! Means: Midsize:27.903 SUV:22.677 Pickup:21.286 Comparing Several Means

11. 11 We have the problem of how to do many comparisons at the same time with some overall measure of confidence in all the conclusions. Statistical methods for dealing with this problem usually have two steps:  An overall test to find any differences among the parameters we want to compare  A detailed follow-up analysis to decide which groups differ and how large the differences are Follow-up analyses can be quite complex; we will look at only the overall test for a difference in several means and examine the data to make follow-up conclusions. Problem of Multiple Comparisons

12. 12 Random sampling always produces chance variations. Any “factor effect” would thus show up in our data as the factor-driven differences plus chance variations (“error”): Data = fit + residual The one-way ANOVA model analyzes situations where chance variations are normally distributed N(0,σ) such that: The One-Way ANOVA Model

13. 13 The ANOVA F Test We want to test the null hypothesis that there are no differences among the means of the populations. The basic conditions for inference are that we have random samples from each population and that the population is Normally distributed. The alternative hypothesis is that there is some difference. That is, not all means are equal. This hypothesis is not one-sided or two- sided. It is “many-sided.” This test is called the analysis of variance F test (ANOVA).

14. 14 Conditions for ANOVA Like all inference procedures, ANOVA is valid only in some circumstances. The conditions under which we can use ANOVA are: Conditions for ANOVA Inference  We have I independent SRSs, one from each population. We measure the same response variable for each sample.  The i th population has a Normal distribution with unknown mean µ i. One-way ANOVA tests the null hypothesis that all population means are the same.  All of the populations have the same standard deviation , whose value is unknown. Conditions for ANOVA Inference  We have I independent SRSs, one from each population. We measure the same response variable for each sample.  The i th population has a Normal distribution with unknown mean µ i. One-way ANOVA tests the null hypothesis that all population means are the same.  All of the populations have the same standard deviation , whose value is unknown. Checking Standard Deviations in ANOVA  The results of the ANOVA F test are approximately correct when the largest sample standard deviation is no more than twice as large as the smallest sample standard deviation. Checking Standard Deviations in ANOVA  The results of the ANOVA F test are approximately correct when the largest sample standard deviation is no more than twice as large as the smallest sample standard deviation.

15. 15 The ANOVA F Statistic To determine statistical significance, we need a test statistic that we can calculate: The ANOVA F Statistic The analysis of variance F statistic for testing the equality of several means has this form: The ANOVA F Statistic The analysis of variance F statistic for testing the equality of several means has this form: F must be zero or positive –F is zero only when all sample means are identical –F gets larger as means move further apart Large values of F are evidence against H 0 : equal means The F test is upper-one-sided

16. 16 F Distributions The F distributions are a family of right-skewed distributions that take only values greater than 0. A specific F distribution is determined by the degrees of freedom of the numerator and denominator of the F statistic. When describing an F distribution, always give the numerator degrees of freedom first. Our brief notation will be F(df1, df2) with df1 degrees of freedom in the numerator and df2 degrees of freedom in the denominator. Degrees of Freedom for the F Test We want to compare the means of I populations. We have an SRS of size n from the i th population, so that the total number of observations in all samples combined is: N = n 1 + n 2 + … + n i If the null hypothesis that all population means are equal is true, the ANOVA F statistic has the F distribution with I – 1 degrees of freedom in the numerator and N – I degrees of freedom in the denominator. Degrees of Freedom for the F Test We want to compare the means of I populations. We have an SRS of size n from the i th population, so that the total number of observations in all samples combined is: N = n 1 + n 2 + … + n i If the null hypothesis that all population means are equal is true, the ANOVA F statistic has the F distribution with I – 1 degrees of freedom in the numerator and N – I degrees of freedom in the denominator.

17. 17 The ANOVA F Test  The measures of variation in the numerator and denominator are mean squares:  Numerator: Mean Square for Groups (MSG)  Denominator: Mean Square for Error (MSE) MSE is also called the pooled sample variance, written as s p 2 (s p is the pooled standard deviation) s p 2 estimates the common variance  2

18. The ANOVA Table Source of variationSum of squares SS DFMean square MS FP valueF crit Among or between “groups” I -1SSG/DFGMSG/MSETail area above F Value of F for  Within groups or “error” N - ISSE/DFE TotalSST=SSG+SSEN – 1 R 2 = SSG/SST√MSE = s p Coefficient of determinationPooled standard deviation The sum of squares represents variation in the data: SST = SSG + SSE. The degrees of freedom reflect the ANOVA model: DFT = DFG + DFE. Data (“Total”) = fit (“Groups”) + residual (“Error”) 18

19. 19 Example Follow-up analysis P-value<.05 significant differences There is significant evidence that the three types of vehicle do not all have the same gas mileage. From the confidence intervals (and looking at the original data), we see that SUVs and pickups have similar fuel economy and both are distinctly poorer than midsize cars.

20. 12.2 Comparing the Means  Contrasts  Multiple Comparisons  Power of the F Test* 20

21. You have calculated a p-value for your ANOVA test. Now what? If you found a significant result, you still need to determine which treatments were different from which.  You can gain insight by looking back at your plots (boxplot, mean ± s).  There are several tests of statistical significance designed specifically for multiple tests. You can choose contrasts, or multiple comparisons.  You can find the confidence interval for each mean  i shown to be significantly different from the others. 21 Introduction

22.  Contrasts can be used only when there are clear expectations BEFORE starting an experiment, and these are reflected in the experimental design. Contrasts are planned comparisons.  Patients are given either drug A, drug B, or a placebo. The three treatments are not symmetrical. The placebo is meant to provide a baseline against which the other drugs can be compared.  Multiple comparisons should be used when there are no justified expectations. Those are pair-wise tests of significance.  We compare gas mileage for eight brands of SUVs. We have no prior knowledge to expect any brand to perform differently from the rest. Pair-wise comparisons should be performed here, but only if an ANOVA test on all eight brands reached statistical significance first. It is NOT appropriate to use a contrast test when suggested comparisons appear only after the data is collected. 22 Introduction

23. Contrasts When an experiment is designed to test a specific hypothesis that some treatments are different from other treatments, we can use contrasts to test for significant differences between these specific treatments.  Contrasts are more powerful than multiple comparisons because they are more specific. They are more able to pick up a significant difference.  You can use a t test on the contrasts or calculate a t confidence interval.  The results are valid regardless of the results of your multiple sample ANOVA test (you are still testing a valid hypothesis). 23

24. To test the null hypothesis H 0 :  = 0 use the t statistic: with degrees of freedom DFE that is associated with s p. The alternative hypothesis can be one- or two-sided. A level C confidence interval for the difference  is: where t* is the critical value defining the middle C% of the t distribution with DFE degrees of freedom. A contrast is a combination of population means of the form where the coefficients a i have sum 0. The corresponding sample contrast is: The standard error of c is: 24 Contrasts

25. Contrasts are not always readily available in statistical software packages (when they are, you need to assign the coefficients “a i ”), or may be limited to comparing each sample to a control. If your software doesn’t provide an option for contrasts, you can test your contrast hypothesis with a regular t test using the formulas we just highlighted. Remember to use the pooled variance and degrees of freedom as they reflect your better estimate of the population variance. Then you can look up your p-value in a table of t distribution. 25 Contrasts

26. Do nematodes affect plant growth? A botanist prepares 16 identical planting pots and adds different numbers of nematodes into the pots. Seedling growth (in mm) is recorded two weeks later. One group contains no nematodes at all. If the botanist planned this group as a baseline/control, then a contrast of all the nematode groups against the control would be valid. 26 Example

27. Contrast of all the nematode groups against the control: Combined contrast hypotheses: H 0 : µ 1 = 1/3 (µ 2 + µ 3 + µ 4 ) vs. H a : µ 1 > 1/3 (µ 2 + µ 3 + µ 4 )  one tailed Contrast coefficients: (+1 −1/3 −1/3 −1/3) or (+3 −1 −1 −1) In Excel: TDIST(3.6,12,1) = tdist(t, df, tails) ≈ 0.002 (p-value). Nematodes result in significantly shorter seedlings (alpha 1%). x i s i G1: 0 nematode10.652.053 G2: 1,000 nematodes10.4251.486 G3: 5,000 nematodes 5.61.244 G4: 10,000 nematodes 5.451.771 27 Example

28.  Planned comparison: H 0 : µ 1 = 1/3 (µ 2 + µ 3 + µ 4 ) vs. H a : µ 1 > 1/3 (µ 2 + µ 3 + µ 4 )  one tailed Contrast coefficients: (+3 −1 −1 −1)  ANOVA: H 0 : all µ i are equal vs. H a : not all µ i are equal  not all µ i are equal Nematodes result in significantly shorter seedlings (alpha 1%). 28 Example

29. Multiple Comparisons Multiple comparison tests are variants on the two-sample t test.  They use the pooled standard deviation s p = √MSE.  The pooled degrees of freedom DFE.  And they compensate for the multiple comparisons. We compute the t statistic for all pairs of means: A given test is significant (µ i and µ j significantly different), when |t ij | ≥ t** (df = DFE). The value of t** depends on which procedure you choose to use. 29

30. The Bonferroni procedure performs a number of pair-wise comparisons with t tests and then multiplies each p-value by the number of comparisons made. This ensures that the probability of making any false rejection among all comparisons made is no greater than the chosen significance level α. As a consequence, the higher the number of pair-wise comparisons you make, the more difficult it will be to show statistical significance for each test. But the chance of committing a type I error also increases with the number of tests made. The Bonferroni procedure lowers the working significance level of each test to compensate for the increased chance of type I errors among all tests performed. The Bonferroni Procedure 30

31. Simultaneous Confidence Intervals We can also calculate simultaneous level C confidence intervals for all pair-wise differences (µ i − µ j ) between population means: s p is the pooled variance, MSE. t** is the t critical value with degrees of freedom DFE = N – I, adjusted for multiple, simultaneous comparisons (e.g., Bonferroni procedure). 31

32. Power* The power, or sensitivity, of a one-way ANOVA is the probability that the test will be able to detect a difference among the groups (i.e., reach statistical significance) when there really is a difference. Estimate the power of your test while designing your experiment to select sample sizes appropriate to detect an amount of difference between means that you deem important.  Too small a sample is a waste of experiment, but too large a sample is also a waste of resources.  A power of about 80% is often suggested. 32

33. Power Computations* ANOVA power is affected by:  The significance level   The sample sizes and number of groups being compared  The differences between group means µ i  The guessed population standard deviation You need to decide what alternative H a you would consider important, detect statistically for the means µ i, and guess the common standard deviation σ (from similar studies or preliminary work). The power computations then require calculating a non-centrality parameter λ, which follows the F distribution with DFG and DFE degrees of freedom to arrive at the power of the test. 33

34. 34 Chapter 12 One-Way Analysis of Variance 12.1 Inference for One-Way Analysis of Variance 12.2 Comparing the Means