Chapter 13 Analysis of Variance (ANOVA) PSY Spring 2003

Chapter 13 One-way Analysis of Variance 2 Major Points An example Requirements The logic Calculations –Unequal sample sizes Cont.

Chapter 13 One-way Analysis of Variance 3 Major Points--cont. Multiple comparisons –Fisher’s LSD –Bonferroni t Assumptions of analysis of variance Magnitude of effect –eta squared –omega squared Review questions

Chapter 13 One-way Analysis of Variance 4 Example Daily caffeine intake (DV) Year in school (IV) –four groups –6 pairs of comparison

Chapter 13 One-way Analysis of Variance 5 Requirements The DV must be quantitative (Interval or Ratio) The IV must be between subjects (so each subject is in one and only one treatment group) The IV must have 2 or more levels.

Chapter 13 One-way Analysis of Variance 6 The F-ratio For the t-test we had t =mean differences between samples t =mean differences between samples standard error of the mean for the F-ratio we use, F =variance (differences) between means F =variance (differences) between means variance (differences) from sampling error

Chapter 13 One-way Analysis of Variance 7 Another silly example Alcohol consumption and driving performance (number of trashed cones) 3 levels of the IV: 1oz, 3oz, 5oz of ETOH N=15, n 1 =5, n 2 =5, n 3 =5 What is the null hypothesis? What is the alternative?

Chapter 13 One-way Analysis of Variance 8 The data

Chapter 13 One-way Analysis of Variance 9 The steps Measure total (or overall) variability –Two components between treatments variability within treatments variability

Chapter 13 One-way Analysis of Variance 10 The steps (continued) Between treatments variability comprised of: –treatment effects –individual differences –experimental error Within treatments variability comprised of: –individual differences –experimental error

Chapter 13 One-way Analysis of Variance 11 The steps (continued) Production of the F-ratio F = Variance between treatment group means Variance within treatment groups OR F = Treatment Effects+Individual Differences+Error Individual Differences+Error

Chapter 13 One-way Analysis of Variance 12 Logic of the Analysis of Variance Null hypothesis: H 0 Population means equal –  1 =       Alternative hypothesis: H 1 –Not all population means equal. Cont.

Chapter 13 One-way Analysis of Variance 13 F-ratio Ratio approximately 1 if null true –Ratio significantly larger than 1 if null false –“approximately 1” can actually be as high as 2 or 3, but not much higher

Chapter 13 One-way Analysis of Variance 14 Epinephrine and Memory Based on Introini-Collison & McGaugh (1986) –Trained mice to go left on Y maze –Injected with 0,.1,.3, or 1.0 mg/kg epinephrine –Next day trained to go right in same Y maze –dep. Var. = # trials to learn reversal More trials indicates better retention of Day 1 Reflects epinephrine’s effect on memory

Chapter 13 One-way Analysis of Variance 15 Grand mean = 3.78

Chapter 13 One-way Analysis of Variance 16 Calculations Start with Sum of Squares (SS) –We need: SS total SS within or sometimes called SS error SS between or sometimes called SS groups Compute degrees of freedom (df ) Compute mean squares and F Cont.

Chapter 13 One-way Analysis of Variance 17 Calculations--cont.

Chapter 13 One-way Analysis of Variance 18 Degrees of Freedom (df ) Number of “observations” free to vary –df total = N - 1 Variability of N observations –df groups = k - 1 variability of g means –df error = k (n - 1) n observations in each group = n - 1 df times k groups

Chapter 13 One-way Analysis of Variance 19 Summary Table

Chapter 13 One-way Analysis of Variance 20 Conclusions The F for groups is significant. –We would obtain an F of this size, when H 0 true, less than one time out of –The difference in group means cannot be explained by random error. –The number of trials to learn reversal depends on level of epinephrine. Cont.

Chapter 13 One-way Analysis of Variance 21 Conclusions--cont. The injection of epinephrine following learning appears to consolidate that learning. High doses may have negative effect.

Chapter 13 One-way Analysis of Variance 22 Unequal Sample Sizes With one-way, no particular problem –Multiply mean deviations by appropriate n i as you go –The problem is more complex with more complex designs, as shown in next chapter. Example from Foa, Rothbaum, Riggs, & Murdock (1991)

Chapter 13 One-way Analysis of Variance 23 Post-Traumatic Stress Disorder Four treatment groups given psychotherapy – –Stress Inoculation Therapy (SIT) Standard techniques for handling stress – –Prolonged exposure (PE) Reviewed the event repeatedly in their mind Cont.

Chapter 13 One-way Analysis of Variance 24 Post-Traumatic Stress Disorder--cont. –Supportive counseling (SC) Standard counseling –Waiting List Control (WL) No treatment

Chapter 13 One-way Analysis of Variance 25 SIT = Stress Inoculation Therapy PE = Prolonged Exposure SC = Supportive Counseling WL = Waiting List Control Grand mean =

Chapter 13 One-way Analysis of Variance 26 Tentative Conclusions Fewer symptoms with SIT and PE than with other two Also considerable variability within treatment groups Is variability among means just a reflection of variability of individuals?

Chapter 13 One-way Analysis of Variance 27 Calculations Almost the same as earlier –Note differences We multiply by n j as we go along. MS error is now a weighted average. Cont.

Chapter 13 One-way Analysis of Variance 28 Calculations--cont.

Chapter 13 One-way Analysis of Variance 29 Summary Table F.05 (3,41) = 2.84

Chapter 13 One-way Analysis of Variance 30 Conclusions F is significant at  =.05 The population means are not all equal Some therapies lead to greater improvement than others. –SIT appears to be most effective.

Chapter 13 One-way Analysis of Variance 31 Multiple Comparisons Significant F only shows that not all groups are equal –We want to know what groups are different. Such procedures are designed to control familywise error rate. –Familywise error rate defined –Contrast with per comparison error rate

Chapter 13 One-way Analysis of Variance 32 More on Error Rates Most tests reduce significance level (  ) for each t test. The more tests we run the more likely we are to make Type I error. –Good reason to hold down number of tests

Chapter 13 One-way Analysis of Variance 33 Fisher’s LSD Procedure Requires significant overall F, or no tests Run standard t tests between pairs of groups. –Often we replace s 2 j or pooled estimate with MS error from overall analysis It is really just a pooled error term, but with more degrees of freedom--pooled across all treatment groups.

Chapter 13 One-way Analysis of Variance 34 Bonferroni t Test Run t tests between pairs of groups, as usual –Hold down number of t tests –Reject if t exceeds critical value in Bonferroni table Works by using a more strict value of  for each comparison Cont.

Chapter 13 One-way Analysis of Variance 35 Bonferroni t--cont. Critical value of  for each test set at.05/c, where c = number of tests run –Assuming familywise  =.05 –e. g. with 3 tests, each t must be significant at.05/3 =.0167 level. With computer printout, just make sure calculated probability <.05/c Necessary table is in the book

Chapter 13 One-way Analysis of Variance 36 Assumptions Assume: –Observations normally distributed within each population –Population variances are equal Homogeneity of variance or homoscedasticity –Observations are independent Cont.

Chapter 13 One-way Analysis of Variance 37 Assumptions--cont. Analysis of variance is generally robust to first two –A robust test is one that is not greatly affected by violations of assumptions.

Chapter 13 One-way Analysis of Variance 38 Magnitude of Effect Eta squared (  2 ) –Easy to calculate –Somewhat biased on the high side –Formula See slide #33 –Percent of variation in the data that can be attributed to treatment differences Cont.

Chapter 13 One-way Analysis of Variance 39 Magnitude of Effect--cont. Omega squared (  2 ) –Much less biased than  2 –Not as intuitive –We adjust both numerator and denominator with MS error –Formula on next slide

Chapter 13 One-way Analysis of Variance 40  2 and  2 for Foa, et al.  2 =.18: 18% of variability in symptoms can be accounted for by treatment  2 =.18: 18% of variability in symptoms can be accounted for by treatment  2 =.12: This is a less biased estimate, and note that it is 33% smaller.  2 =.12: This is a less biased estimate, and note that it is 33% smaller.

Chapter 13 One-way Analysis of Variance 41 Review Questions Why is it called the analysis of “variance” and not the analysis of “means?” What do we compare to create our test? Why would large values of F lead to rejection of H 0 ? What do we do differently with unequal n ’s? Cont.

Chapter 13 One-way Analysis of Variance 42 Review Questions--cont. What is the per comparison error rate? Why would the familywise error rate generally be larger than.05 unless it is controlled? Most instructors hate Fisher’s LSD. Can you guess why? –Why should they not hate it? Cont.

Chapter 13 One-way Analysis of Variance 43 Review Questions--cont. How does the Bonferroni test work? What assumptions does the analysis of variance require? What does “robust test” mean? What do we mean by “magnitude of effect?” What do you know if  2 is.60?