1. Lecture 10 PY 427 Statistics 1 Fall 2006 Kin Ching Kong, Ph.D
Chicago School of Professional Psychology
Lecture 10
Kin Ching Kong, Ph.D

2. Agenda Analysis of Variance (continue) The Distribution of F-Ratios
Review Intro. to ANOVA
Hypotheses for ANOVA
The Test Statistic for ANOVA: F
The Logic of Analysis of Variance
ANOVA Notation & Formulas
The Distribution of F-Ratios
Hypothesis Testing
Measuring Effect Size
Assumptions for the independent-measure ANOVA
Post Hoc Tests

3. Analysis of Variance (ANOVA)
ANOVA is used to compare two or more means
The Question:
Does the mean differences observed among the samples reflect mean differences among the populations?
Two Possibilities:
There really are no differences in the population means, the observed differences are due to chance (sampling error).
The population means are truly different, and are partly responsible for differences in the sample means.
Figure 13.1

4. Hypotheses for ANOVA
The experiment: learning performance under three temperature: 50o, 70o, 90o
Design: single-factor, between-subject (or independent-measures.
The Hypotheses:
H0: m1 = m2 = m3 (no differences in pop. means)
H1: At least one pop. mean is different from the others, (or not all the pop. means are equal).
There are many possible specific alternative hypotheses (e.g. all 3 means are different, first two means are identical but the third is different, the first is different and last two identical, etc.)
So far we talked about inferential statistics using one sample to make inferences about an unknown population.

5. The Test Statistic for ANOVA
The Test Statistic for t tests:
t = obtained difference between sample means
difference expected by chance (error)
The Test Statistic for ANOVA, F-ratio:
F = variance (differences) between sample means
variance (differences) expected by chance (error)
The F-ratio is based on variance rather than means
Problem: how to define & calculate mean differences when there are more than two means.
Solution: use variance to define and measure the size of differences among the sample means.
E.g M1 = 20, M2 = 30, M3 =35 s2 = 58.33
M1 = 28, M2 = 30, M3 = s2 = 2.33

6. The Logic of Analysis of Variance, The Data
The experiment: learning under 3 different temperature
The design: single factor between-subjects
The data: DV = # of problems solved correctly.
Temp. 50O
Temp. 70O 4 1 3 2 6
M = 1
M = 4

7. The Logic of Analysis of Variance
The Goal of ANOVA
Measure the amount of variability in a data set and explain where it comes from.
Step I: Measure Total Variance
The variability in the whole data set (scores from all samples combined)
Step II: Partition Total Variance into two components:
Between-Treatment Variance
Differences between treatment conditions
Within-Treatment Variance:
Differences within each treatment condition

8. The Between-Treatments Variance
Measures how much difference exists between the treatment conditions (i.e. differences among treatment means).
Two Sources For The Differences Between Treatments:
Treatment effects
Chance (unplanned & unpredictable difference)
Individual differences
Experimental error (Measurement error)
To Demonstrate There Really Is A Treatment Effect:
Show that differences between treatments are bigger than would be expected by chance alone.

9. The Within-Treatment Variance
Measures differences due to chance, or when there is no treatment effect, H0 is true.
Figure 13.2 Analysis of Variance

10. The F-Ratio: Test Statistic for ANOVA
The F-Ratio compares the two component variance:
F = Variance between treatments
Variance within treatments
= actual differences between treatments
differences expected with no treatment effect
F = treatment effect + differences due to chance
differences due to chance (error)
When there is no treatment effect:
F = differences due to chance F is close to 1
differences due to chance
When there is a treatment effect:
The numerator significantly > the denominator
F significantly > 1

11. ANOVA Notations SX2 = 106 G = 30 N = 15 k = 3 T1 = 5 T2 = 20 T3 = 5
SS1 = SS2 = 6 SS3 = 4
n1 = n2 = n3 = 5
M1 = M2 = M3 = 1
Temp. 50O
Temp. 70O 4 1 3 2 6
M = 1
M = 4

12. ANOVA Notations (Continue)
k = number of treatment conditions (or number of levels of a factor)
ni = number of scores in treatment i (i = 1 to k)
N = total number of scores in the entire study.
Ti = The total (SX) for treatment condition I
G = The Grand Total = SX for all the scores, or G = ST
For a one sample t test, the statistic is M, so the standard error is sM

13. ANOVA Formulas
ANOVA Summary Table:
Source SS df MS
Between treatments F = MSbetween
Within treatments MSwithin
Total
F = Variance between treatments
Variance within treatments
Variance, s2 = SS/df =MS
F = MSbetween
MSwithin
To fill in the ANOVA summary table, need to calculate nine values: 3 SS, 3 df, 2 MS and F

14. ANOVA Formulas: SS Total Sum of Squares:
SStotal = SX2 – (SX)2 = SX2 – G2
N N
For our example:
SStotal = SX2 – G2 = /15 = 106 – 60 = 46 N
Within-Treatment Sum of Squares:
SSwithin = SSSwithin each treatment
For our example: SSwithin = = 16
Between-Treatment Sum of Squares:
SSbetween = S T2 - G2
n N
SSbetween = S T2 - G2 = – 302 = =30
n N

15. ANOVA Formulas, df Total Degrees of Freedom:
dftotal = N - 1
For our example, dftotal = 15 – 1 = 14
Within-Treatment Degrees of Freedom:
dfwithin = Sdfin each treatment = S(n-1) = N - k
For our example, dfwithin = 15 – 3 = 12
Between-Treatment Degrees of Freedom:
dfbetween = k – 1
For our example, dfbetween = 3 – 1 = 2

16. ANOVA Formulas, MS & F-Ratio
Variance Between-Treatment, MSbetween:
s2 = MSbetween = SSbetween/dfbetween
For our example, MSbetween = 30/2 = 15
Variance Within-Treatment, MSwithin:
s2 = MSwithin = SSwithin/dfwithin
For our example, MSwithin = 16/12 = 1.33
The F-Ratio:
F = MSbetween
MSwithin
For our example: F = 15/1.33 = 11.28

17. ANOVA Formulas Source SS df MS ANOVA Summary Table:
Between treatments F = 11.28
Within treatments
Total

18. The Distribution of F-Ratios
Two characteristics of F values:
F values are always positive because variances are always positive.
When H0 is true, the numerator and denominator of the F-ratio estimate the same variance, thus, the ratio should be near 1. In other words, the distribution of F-ratios should pile up around 1.00
The Distribution of F-ratios:
Cut off at zero (all positive values)
Piles up around 1.00
Tapers off to the right.
The exact shape of the F distribution depends on the df’s in the two variances.
Figure 13.6 of your book

19. The F Distribution Table
Table B.4
Find df of the numerator in first row.
Find df of the denominator in first column
The intersection of these two df’s is a pair of numbers:
The smaller number is the critical value for a = .05
The larger number is the critical value for a = .01
Table 13.3
E.g. F = 4.18 with df = 2, 15. Is this value sufficient to reject H0 with a = .05? a =.01?

20. Hypothesis Testing (the experiment)
Research Goal:
Evaluate the effectiveness of three pain relievers (A, B, C) and a placebo.
Experiment:
Participants:
four groups, n = 5 in each group
Treatment (IV):
Drug A, B, C and placebo
Design:
Single-factor, repeated-measures
Dependent Variable (DV):
Amount of time participants can withstand a painfully hot stimulus.

21. Hypothesis Testing (Data)
SX2 = 262
T = 5 T = T = T = 25
SS = 8 SS = SS = SS = 10
Placebo
Drug A
Drug B
Drug C 3 4 6 7 2 1 5

22. ANOVA Summary Table ANOVA Summary Table: Source SS df MS
Between treatments F =
Within treatments
Total

23. ANOVA Calculations dftotal = N – 1 = 20 – 1 = 19
dfbetween = k – 1 = 4 – 1 =3
dfwithin = N – k = 16
SStotal = SX2 – G2/N = 262 – 602/20 = 82
SSwithin = SSSinside each treatment= = 32
SSbetween = S T2/n – G2/N
= 52/ / /5 +252/5 – 602/20
= 50
MSbetween = SSbetween/ dfbetween = 50/3 = 16.67
MSwithin = SSwithin/ dfwithin = 32/16 = 2.00
F = MSbetween/ MSwithin = 16.67/2.00 = 8.33

24. ANOVA Formulas Source SS df MS ANOVA Summary Table:
Between treatments F = 8.33
Within treatments
Total

25. Hypothesis Testing with ANOVA
Step 1: State the Hypotheses:
H0: m1 = m2 = m3 = m4 (there is no treatment effect)
H1: At least one of the treatment means are different.
The level of significant is set at a = .05
Step 2: Locate the Critical Region:
df = 3, 16, Fcritical = Figure 13.7
Step 3: Calculate the test statistic:
F = MSbetween / MSwithin = 8.33
Step 4: Make a decision:
Since the test statistic, F = 8.33 falls in the critical region, reject H0 and concludes that there is a significant treatment effect.

26. Measuring Effect Size for ANOVA
A significant difference
Means that the difference observed in the samples is very unlikely to have occurred just by chance.
Statistical significant does not necessarily mean large effect.
Measuring effect size for ANOVA:
r2: the percentage of variance accounted for by treatment
r2 = SSbetween
SStotal
In published reports, the r2 value for ANOVA is usually call h2 (the Greek letter eta squared)
For our example, h2 = 50/82 = 0.61
The pooled variance is used instead of the individual sample variances.

27. Assumptions Assumptions for the Independent-Measure ANOVA:
The observations within each sample must be independent.
The populations from which the samples are selected must be normal.
The populations from which the samples are selected must have equal variances (homogeneity of variance)

28. Post Hoc Tests, Intro A significant F-ratio:
Indicate that a significant difference exit, that not all the means are equal.
Does not indicate which means are different and which are not.
Example:
M1 = 3, M2 = 5, M3 = 10
M2 – M1 = 2, M3 – M2 = 5, M3 – M1 = 7
A significant F indicate that at least one of these differences are significant, M3 = M1, but what about the other two?
Post hoc tests are used to find out which of these difference are significant.

29. Post Hoc Tests & Type I Error
are additional hypothesis test that are done after an analysis of variance revealed a significant difference
They are performed to determine exactly which mean differences are significant and which are not.
Post hoc test and Type I Error
Post hoc tests compare two means at a time, i.e. pairwise comparisons.
The process involve performing a series of separate hypothesis tests.
Each of these tests includes the risk of a Type I error
With more tests, the risk of a type I error accumulates
Experimentwise alpha level: the overall probability of a Type I Error that accumulates over a series of separate hypothesis tests.

30. Post Hoc Tests, Planned Comparisons
Controlling Type I Error (experimentwise alpha level)
Whenever more than one test is done, need to be concerned about the experimentwise Type I Error.
Planned Comparisons
Planned Comparisons: specific mean differences are predicted by specific hypotheses before the study is conducted.
Because a few specific comparisons were planned before the data were collected, many statisticians argue that planned comparisons can be conducted with a standard alpha, without concern about inflating the risk of a Type I error.
Dunn Test: It is often recommended that researchers protect against an inflated alpha level by dividing alpha equally among the planned comparisons.

31. Post Hoc Tests, Tukey’s HSD
Unplanned Comparisons
sifting through the data by conducting a large number of comparisons.
Tukey’s Honestly Significant Difference (HSD)
Compute a single value (HSD) that determines the minimum difference that is necessary for significance.
If a pairwise difference is > Tukey’s HSD, you conclude that there is a significant difference between the two means.
HSD = q
n = number of scores in each treatment, Tukey’s HSD test requires equal n’s
Table B.5 to find value of q. (k = number of treatment conditions, df error term = df for the denominator of the F-ratio)

32. Post Hoc Tests, Tukey’s HSD, Example
M1 = 3.00 M2 = 5.44 M3 = 7.00
ANOVA Summary Table
Source SS df MS
Between F (2, 24) = 9.15
Within
Total
HSD = q = = 2.36
M2 – M1 = 2.44, significant
M3 – M1 = 4.00, significant
M3 – M2 = 1.56, nonsignificant