Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 1 Chapter 12: One-Way Independent ANOVA What type of therapy is best for alleviating social anxiety? Suppose that the options are: counseling, group therapy, and systematic desensitization. Why not just do three t tests, as represented by the following three null hypotheses ? 1)μ co = μ gt 2)μ co = μ sd 3)μ gt = μ sd Because this can lead to too many Type I errors!

Chapter 12For Explaining Psychological Statistics, 4th ed. by B. Cohen 2 F Ratio for ANOVA –When H 0 is true for a one-way ANOVA (i.e., there is one IV with multiple levels), the F ratio (see below) follows a probability distribution called the F distribution (named after Sir Ronald Fisher). –A ratio will follow the F distribution when both the numerator and denominator are independent estimates of the same population variance. –How is the denominator of the preceding formula an estimate of the population variance? See next slide.

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 3 F Ratio for ANOVA (cont.) MS within (the within-group variance estimate) is found by pooling the variances of all of the groups in the study (just as the two group variances were pooled for the t test). –We must assume homogeneity of variance (HOV) to justify pooling the variances. –Given that all of the population variances are the same, MS within (called just MS w ) gives the best estimate of it. How is the numerator of the F ratio in a one-way ANOVA also an estimate of the population variance? –This is much less obvious, but is easier to describe when all of the groups are the same size, n. –In the case of equal-sized groups, MS between is given by the following formula (see next slide).

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 4 F Ratio for ANOVA (cont.) –Note that the symbol following n repres- ents the unbiased variance of the means of all of the groups in the study. –By squaring the standard error of the mean, we know that: –Multiplying both sides by n, we find that: –When H 0 for the ANOVA is true, MS Bet, which is the unbiased variance of the group means multiplied by n (in the equal- sized groups case) is an estimate of the population variance. –Thus, when H 0 is true, MS W and MS Bet are independent estimates of σ 2, and therefore their ratio follows the F distribution.

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 5 F Ratio for ANOVA (cont.) –What are the sources of variability in the denominator of the F ratio? Individual differences Errors in measurement of the dependent variable Fluctuations in the conditions under which subjects are measured Thus the denominator, MS w, is also called MS error, and is therefore referred to as the error term of the ANOVA –What are the sources of variability in the numerator of the F ratio? All of the above sources of error variance, plus … The effect of the experimental manipula- tion, or mean differences among pre- existing groups. This effect is expected to be zero when the null hypothesis is true, in which case the numerator and denominator of the F ratio are expected to be approximately equal.

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 6 Testing the F Ratio for Significance ANOVA usually involves a one-tailed test, in the sense that all of α (alpha) is placed in the positive (rightmost) tail of the F distribution. –The negative (leftmost) tail represents small F ratios (F << 1.0) that occur when the group means are relatively close together. Therefore, F ratios in this tail represent results that are consistent with the null hypothesis, and do not lead to rejections of H 0. –The positive tail contains large F ratios that represent a relatively large spread of the sample means, and therefore may lead to rejection of H 0. –To find critical values for the F distribution in a table, you must know the degrees of freedom (dfs) associated with your calcul- ated F ratio.

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 7 The Shape of the F Distribution –Tends to be positively skewed. –Depends on dfs for both the numerator and denominator of the F ratio. As the df’s increase, the F distri-bution becomes less skewed, and eventually becomes indistinguishable from the normal distribution. Degrees of Freedom for ANOVA –Numerator: df Bet = k – 1 –Denominator: df w = N T – k –Total: df tot = N T – 1 (= N T – k + k – 1) where k equals the number of groups, and N T equals the sum of all the group sizes (i.e., the total number of participants in the study)

Chapter 12For Explaining Psychological Statistics, 4th ed. by B. Cohen 8 Try this example… 3 Types of Learning where (i.e., the mean of all the scores in the study) *Note: The formulas shown above are for the case of equal-sized samples only. Types of Learning Group Instruction Only One-on-One Instruction Computer Instruction Mean5106 s N15

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 9 Answer to the Types of Learning Example Critical values of F: F.05 (2, 42) = 3.22 (approx.) F.01 (2, 42) = 5.15 (approx.)

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 10 Now Try This Example (Using Unequal Sample Sizes)… Are there differences in satisfaction among different academic majors?

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen. 11 Answer to the Academic Majors Example Critical value of F: F.05 (2, 11) = 3.98 F calc.05, and the results must be declared not significant (the null hypothesis cannot be rejected). However, this decision could be a Type II error.

Chapter 12 For Explaining Psychological Statistics, 4th ed. by B. Cohen 12 Major Assumptions of ANOVA –Independent random sampling If using preexisting populations, it is critical to the validity of the study to obtain truly random samples. If using an experimental manipulation, it is common to collect a single sample of convenience, and then randomly assign participants to conditions. –Normal distribution of the DV ANOVA is said to be fairly robust with respect to this assumption, unless the sample sizes are quite small. When dealing with small ns, and very non- normal distributions, consider a nonpara- metric alternative to ANOVA, such as the Kruskal-Wallis H Test. –Homogeneity of Variance (HOV) Not considered an issue when all of the samples are the same size. Rule of thumb: If no sample variance is more than twice as large as another, and no sample is more than 1½ times as large as another, HOV can be safely assumed.

Chapter 12For Explaining Psychological Statistics, 4th ed. by B. Cohen 13 Power and Effect Size for ANOVA –Power is found from a noncentral F distribution, which is centered on an expected F ratio (F A ) –The calculated F for a one-way ANOVA can be expressed as: –Effect size F is the measure of effect size that is the multigroup analog of g (Cohen’s d). Its formula is the same as F above, without the n The corresponding effect size in the population is represented by (bold) f, and is given by the following formula:

Chapter 12For Explaining Psychological Statistics, 4th ed. by B. Cohen 14 f may be estimated from the F ratio calculated for a previous study, using the following formula: Where k is the number of groups in the previous study, and n was the size of each group (note: use the harmonic mean for unequal sample sizes). Or by using the conventional values established by J. Cohen : f =.1 small f =.25 medium f =.4 large

Chapter 12For Explaining Psychological Statistics, 4th ed. by B. Cohen 15 Looking up power in a table requires a measure called phi (Φ) that combines the estimate of population effect size with the planned sample size, according to the following formula: –Φ plays same role for ANOVA that δ does for the t test If you know the value of Φ that you are aiming for, and you have an estimate of the population effect size, you can use the following formula to estimate the sample size you will need to attain the level of power implied by your desired value for Φ:

Chapter 12For Explaining Psychological Statistics, 4th ed. by B. Cohen 16 Varieties of One-Way ANOVA –The levels of your factor (IV) may have been created for the experiment, or represent pre- existing groups. –Fixed vs. random effects The levels of your IV can be specific (fixed) ones, or just a (random) sampling from many possible levels. Random Effects ANOVA: When the levels of the IV are selected at random from (and representative of) a larger set. Fixed Effects ANOVA: When the experimenter is specifically interested in the levels chosen and not concerned with extrapolating to untested levels.