1. Analysis of Variance (ANOVA) Quantitative Methods in HPELS 440:210

2. Agenda Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions

3. Introduction Recall  There are two possible scenarios when obtaining two sets of data for comparison:  Independent samples: The data in the first sample is completely INDEPENDENT from the data in the second sample.  Dependent/Related samples: The two sets of data are DEPENDENT on one another. There is a relationship between the two sets of data.

4. Introduction Three or more data sets?  If the three or more sets of data are independent of one another  Analysis of Variance (ANOVA)  If the three or more sets of data are dependent on one another  Repeated Measures ANOVA

5. Introduction: Terminology Factor: Synonym of independent variable Level: The treatment conditions that make up the factor or independent variable Example: What is the effect of grade (1 st, 2 nd, 3 rd ) on IQ?  Dependent variable: IQ  Factor: Grade  Levels (3): 1 st, 2 nd and 3 rd grades

6. Introduction: Terminology Between-Treatment Variance: Variance between the treatments/levels As the between-treatment variance increases:  The statistic increases  The p-value decreases Greater chance of rejecting the H 0

7. Introduction: Terminology Within-Treatment Variance: Variance within the treatments/levels As the within-treatment variance increases:  The statistic decreases  The p-value increases Lesser chance of rejecting the H 0

8. Recall the Independent-Measures t-Test If there was a large difference between the means (between variance)  t got bigger Why?  t = M 1 -M 2 / s (M1-M2) The t formula can be thought of as a ratio of:  Between variance (M 1 -M 2 )  Within variance (s (M1-M2) ) Several Scenarios can occur

9. -Small between variance -Large within variance -t = BV / WV = near zero value Accept or reject the H 0

10. -Large between variance -Large within variance -t = BV / WV = near value of 1.0 Accept or reject the H 0

11. -Small between variance -Small within variance -t = BV / WV = near value of 1.0 Accept or reject the H 0

12. -Large between variance -Small within variance -t = BV / WV = greater than 1.0 Accept or reject the H 0

13. Introduction  The F-Ratio ANOVA is a ratio of between variance and within variance Distinction: Three or more groups

14. The F Distribution Plot all possible F-ratios  F distribution There is a family of F distributions As df increases, the distribution becomes more narrow F-ratios are always positive in value  Computed with two variances  Variances are always positive! F distribution is skewed Most values cluster around 1.0 Figure 13.8 (p 413)

16. Agenda Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions

17. ANOVA Statistical Notation:  k = number of treatment conditions (levels)  n x = number of samples per treatment level  N = total number of samples N = kn if sample sizes are equal  T x =  X for any given treatment level  G =  T  MS = mean square = variance

18. ANOVA Formula Considerations:  SS between =  T 2 /n – G 2 /N  SS within =  SS inside each treatment  SS total = SS within + SS between SS total =  X 2 – G 2 /N

19. ANOVA Formula Considerations:  df total = N – 1  df between = k – 1  df within =  (n – 1) df within =  df in each treatment

20. ANOVA Formula Considerations:  MS between = s 2 between = SS between / df between  MS within = s 2 within = SS within / df within  F = MS between / MS within

21. Independent-Measures Designs Static-Group Comparison Design:  Administer treatment to two or more groups and perform posttest  Perform posttest to control group  Compare groups X1O X2O O

22. Independent-Measures Designs Quasi-Experimental Pretest Posttest Control Group Design:  Perform pretest on three or more groups  Administer treatments to treatment groups  Perform posttests on all groups  Compare delta (Δ) scores OX1O  Δ OX2O  Δ OO  Δ

23. Independent-Measures Designs Randomized Pretest Posttest Control Group Design:  Randomly select subjects from three or more populations  Perform pretest on all groups  Administer treatments to treatment groups  Perform posttests on all groups  Compare delta (Δ) scores ROX1O  Δ ROX2O  Δ ROO  Δ

24. Agenda Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions

25. Hypothesis Test: ANOVA Example 13.1 (p 415) Overview:  Researchers are interested in the effectiveness different pain relievers (A, B and C) compared placebo (D)  N = 20 randomly assigned to the four treatments (n = 5)  Amount of time (s) each subject could withstand a painfully hot stimulus was measured

26. Hypothesis Test: ANOVA Questions:  What is the experimental design?  What is the independent variable/factor?  How many levels are there?  What is the dependent variable?

27. Step 1: State Hypotheses Non-Directional H 0 : µ A = µ B = µ C = µ D H 1 : At least one mean is different than the others Directional? Too many too list Step 2: Set Criteria Alpha (  ) = 0.05 Critical Value: Use F Distribution Table Appendix B.4 (p 693) Information Needed: df between = k – 1 df within =  (n – 1)

28. Step 3: Collect Data and Calculate Statistic Total Sum of Squares SS total =  X 2 – G 2 /N SS total = 262 – 60 2 /20 SS total = 262 - 180 SS total = 82 Sum of Squares Between SS between =  T 2 /n –  G 2 /N SS between = 5 2 /5+10 2 /5+20 2 /5+25 2 /5 – 60 2 /20 SS between = (5+20+80+125) - 180 SS between = 50 Sum of Squares Within SS within =  SS inside each treatment SS within = 8+8+6+10 SS within = 32

29. Step 3: Collect Data and Calculate Statistic Mean Square Between MS between = SS between / df between MS between = 50 / 3 MS between = 16.67 Mean Square Within MS within = SS within / df within MS within = 32/16 MS within = 2 F-Ratio F = MS between / MS within F = 16.67 / 2 F = 8.33 Step 4: Make Decision

30. Agenda Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions

31. Post Hoc Analysis What ANOVA tells us:  Rejection of the H 0 tells you that there is a high PROBABILITY that AT LEAST ONE difference exists SOMEWHERE What ANOVA doesn’t tell us:  Where the differences lie Post hoc analysis is needed to determine which mean(s) is(are) different

32. Post Hoc Analysis Post Hoc Tests: Additional hypothesis tests performed after a significant ANOVA test to determine where the differences lie. Post hoc analysis IS NOT PERFORMED unless the initial ANOVA H 0 was rejected!

33. Post Hoc Analysis  Type I Error Type I error: Rejection of a true H 0 Pairwise comparisons: Multiple post hoc tests comparing the means of all “pairwise combinations” Problem: Each post hoc hypothesis test has chance of type I error As multiple tests are performed, the chance of type I error accumulates Experimentwise alpha level: Overall probability of type I error that accumulates over a series of pairwise post hoc hypothesis tests How is this accumulation of type I error controlled?

34. Two Methods Bonferonni or Dunn’s Method:  Perform multiple t-tests of desired comparisons or contrasts  Make decision relative to  / # of tests  This reduction of alpha will control for the inflation of type I error Specific post hoc tests:  Note: There are many different post hoc tests that can be used  Our book only covers two (Tukey and Scheffe)

35. Tukey’s Honestly Significant Difference (HSD) Test Overview:  Computes a single value that determines the minimum difference (HSD) between any two means necessary for rejection of the H 0  Compares the HSD value to all of the contrast results  If the contrast result exceeds the HSD, the H 0 of that particular contrast is rejected

36. Tukey’s HSD  Calculation Formulas: Equal sample sizes  HSD = q√MS within / n Unequal sample sizes  HSD = q√(MS within /2)(1/n 1 +1/n 2 )

37. Tukey’s HSD  Calculations Formula Considerations: q = value found in Table B.5 (p 696)  Left column: df within  Top row: k treatments  Body: Regular font:  = 0.05 Bold font:  = 0.01  MS within = value from ANOVA calculation  n = number of subjects in each treatment Example 13.5 (p 427)

38. Step 1: State Hypotheses Null H 0 : µ A = µ B H 0 : µ A = µ C H 0 : µ B = µ C Alternative H 1 : µ A  µ B H 1 : µ A  µ C H 1 : µ B  µ C Step 2: Set Criteria Alpha (  ) = 0.05 Step 3: Calculate Statistic Get q from Table B.5  Information needed: df within = 24 k = 3  = 0.05 q = 3.53

39. Calculate Tukey’s HSD Value HSD = q  MS within / n HSD = 3.53  4 / 9 HSD = 2.36 Step 4: Make Decision: A significantly greater than B  M A – M B = 2.44 > 2.36 A significantly greater than C  M A – M C = 4.00 > 2.36 B not significantly different than C  M B – M C = 1.56 < 2.36 Table 13.6

40. Scheffe Overview:  Most conservative/cautious of all post hoc tests  Uses an F-ratio (like ANOVA) on only two treatments Controls for type I error:  Uses k value from the original ANOVA thus the numerator of the F-ratio for the Scheffe test is k – 1  Uses same critical value used for the ANOVA Calculation of Scheffe is identical to the ANOVA however:  SS between uses the two means of interest Example 13.6 (p 428)

41. Step 1: State Hypotheses Null H 0 : µ A = µ B H 0 : µ A = µ C H 0 : µ B = µ C Alternative H 1 : µ A  µ B H 1 : µ A  µ C H 1 : µ B  µ C Step 2: Set Criteria Alpha (  ) = 0.05 Step 3: Calculate Statistic Sum of squares between: SS between =  T 2 /n – G 2 /N SS between = (27 2 /9 + 49 2 /9) – 76 2 /18 SS between = (81+266.78) – 320.89 SS between = 26.89 SS within from original ANOVA = 96 Critical Value  3.40 df between = 2 df within = 24  = 0.05 Mean square between and within MS between = SS between /df between MS between = 26.89 / 2 = 13.45 MS within from original ANOVA = 4 F = MS between / MS within F = 13.45 / 4 F = 3.36

42. F = MS between / MS within F = 13.45 / 4 F = 3.36 Step 4: Make Decision F = 3.36 < 3.40 (critical value) Accept or reject? Repeat for the other two contrasts: H 0 : µ A = µ C H 0 : µ B = µ C df = 2, 24 3.40

43. Agenda Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions

44. Instat Type dependent variable data from the three or more samples into one column:  Label column appropriately In a second column, type in the grouping variable (independent variable) next to each data point:  Label column appropriately  Convert the grouping column into a “factor” column: Highlight the grouping column. Choose “Manage” Choose “Column Properties” Choose “Factor” Select the appropriate column to be converted Indicate the number of levels in the factor Click OK

45. Instat Choose “Statistics”  Choose “Analysis of Variance”  Choose “One-Way”  Y-Variate: Choose the dependent variable  Factor: Choose the factor column or grouping/independent variable  Plots: Not necessary to choose any  Click OK.  Interpret the p-value!!! Post Hoc Analysis:  Perform multiple Independent-Measures t-Tests with the Bonferonni/Dunn correction method

46. Reporting ANOVA Results Information to include:  Value of the F statistic  Degrees of freedom: Between: k – 1 Within:  (n – 1)  p-value Examples:  A significant treatment effect was observed (F(2, 24) = 8.33, p = 0.02)

47. Reporting ANOVA Results An ANOVA summary table is often included SourceSSdfMS Between50316.67F = 8.33 Within32162 Total8219

48. Agenda Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions

49. Assumptions of ANOVA Independent Observations Normal Distribution Scale of Measurement  Interval or ratio Equal variances (homogeneity)

50. Violation of Assumptions Nonparametric Version  Kruskall-Wallis Test (Chapter 17) When to use the Kruskall-Wallis Test:  Independent-Measures design with three or more groups  Scale of measurement assumption violation: Ordinal data  Normality assumption violation: Regardless of scale of measurement

51. Textbook Assignment Problems: 3, 5, 17a, 21