1. Lecture 13 Multiple comparisons for one-way ANOVA (Chapter 15.7)
Analysis of Variance Experimental Designs (Chapter 15.3)

2. 15.7 Multiple Comparisons
When the null hypothesis is rejected, it may be desirable to find which mean(s) is (are) different, and how they rank.
Three statistical inference procedures, geared at doing this, are presented:
Fisher’s least significant difference (LSD) method
Bonferroni adjustment to Fisher’s LSD
Tukey’s multiple comparison method

3. Example 15.1
Sample means:
Does the quality strategy have a higher mean sales than the other two strategies?
Do the quality and price strategies have a higher mean than the convenience strategy?
Does the price strategy have a smaller mean sales than quality but a higher mean than convenience?
• Pairwise comparison: Are two population means different?

4. Fisher Least Significant Different (LSD) Method
This method builds on the equal variances t-test of the difference between two means.
The test statistic is improved by using MSE rather than sp2.
We conclude that mi and mj differ (at a% significance level if > LSD, where

5. Multiple Comparisons Problem
A hypothetical study of the effect of birth control pills is done.
Two groups of women (one taking birth controls, the other not) are followed and 20 variables are recorded for each subject such as blood pressure, psychological and medical problems.
After the study, two-sample t-tests are performed for each variable and it is found that one null hypothesis is rejected. Women taking birth pills have higher incidences of depression at the 5% significance level (the p-value equals .02).
Does this provide strong evidence that women taking birth control pills are more likely to be depressed?

6. Experimentwise Type I error rate (aE) versus Comparisonwise Type I error rate
The comparisonwise Type I error rate is the probability of committing a Type I error for one pairwise comparison.
The experimentwise Type I error rate ( ) is the probability of committing at least one Type I error when C tests are done and all null hypotheses are true.
For a one-way ANOVA, there are k(k-1)/2 pairwise comparisons (k=number of populations)
If the comparisons are not planned in advance and chosen after looking at the data, the experimentwise Type I error rate is the more appropriate one to look at.

7. Experimentwise Error Rate
The expected number of Type I errors if C tests are done at significance level each is
If C independent tests are done,
aE = 1-(1 – a)C
The Bonferroni adjustment determines the required Type I error probability per test (a) , to secure a pre-determined overall aE.

8. Bonferroni Adjustment
Suppose we carry out C tests at significance level
If the null hypothesis for each test is true, the probability that we will falsely reject at least one hypothesis is at most
Thus, if we carry out C tests at significance level , the experimentwise Type I error rate is at most

9. Bonferroni Adjustment for ANOVA
The procedure:
Compute the number of pairwise comparisons (C) [all: C=k(k-1)/2], where k is the number of populations.
Set a = aE/C, where aE is the true probability of making at least one Type I error (called experimentwise Type I error).
We conclude that mi and mj differ at a/C% significance level (experimentwise error rate at most ) if

10. Fisher and Bonferroni Methods
Example continued
Rank the effectiveness of the marketing strategies (based on mean weekly sales).
Use the Fisher’s method, and the Bonferroni adjustment method
Solution (the Fisher’s method)
The sample mean sales were , 653.0,
Then,

11. Fisher and Bonferroni Methods
Solution (the Bonferroni adjustment)
We calculate C=k(k-1)/2 to be 3(2)/2 = 3.
We set a = .05/3 = .0167, thus t.0167/2, 60-3 = (Excel).
Again, the significant difference is between m1 and m2.

12. Tukey Multiple Comparisons
The test procedure:
Assumes equal number of obs. per populations.
Find a critical number w as follows:
k = the number of populations
n =degrees of freedom = n - k
ng = number of observations per population
a = significance level
qa(k,n) = a critical value obtained from the studentized range table (app. B17/18)

13. Tukey Multiple Comparisons
Select a pair of means. Calculate the difference between the larger and the smaller mean.
If there is sufficient evidence to conclude that mmax > mmin .
Repeat this procedure for each pair of samples. Rank the means if possible.
If the sample sizes are not extremely different, we can use the above procedure with ng calculated as the harmonic mean of
the sample sizes.

14. Tukey Multiple Comparisons
Example continued We had three populations (three marketing strategies). K = 3,
Sample sizes were equal. n1 = n2 = n3 = 20, n = n-k = 60-3 = 57, MSE = 8894.
Take q.05(3,60) from the table: 3.40.
Population
Sales - City 1
Sales - City 2
Sales - City 3
Mean
577.55
653
698.65
City 1 vs. City 2: = 75.45
City 1 vs. City 3: = 31.1
City 2 vs. City 3: = 44.35

15. 15.3 Analysis of Variance Experimental Designs
Several elements may distinguish between one experimental design and another:
The number of factors (1-way, 2-way, 3-way,… ANOVA).
The number of factor levels.
Independent samples vs. randomized blocks
Fixed vs. random effects
These concepts will be explained in this lecture.

16. Number of factors, levels
Example: 15.1, modified
Methods of marketing: price, convenience, quality => first factor with 3 levels
Medium: advertise on TV vs. in newspapers => second factor with 2 levels
This is a factorial experiment with two “crossed factors” if all 6 possibilities are sampled or experimented with.
It will be analyzed with a “2-way ANOVA”. (The book got this term wrong.)

17. One - way ANOVA Single factor Two - way ANOVA Two factors
Response
Response
Treatment 3 (level 1)
Treatment 2 (level 2)
Treatment 1 (level 3)
Level 3
Level2
Factor A
Level 1
Level2
Level 1
Factor B

18. Randomized blocks
This is something between 1-way and 2-way ANOVA: a generalization of matched pairs when there are more than 2 levels.
Groups of matched observations are collected in blocks, in order to remove the effects of unwanted variability. => We improve the chances of detecting the variability of interest.
Blocks are like a second factor => 2-way ANOVA is used for analysis
Ideally, assignment to levels within blocks is randomized, to permit causal inference.

19. Randomized blocks (cont.)
Example: expand 13.03
Starting salaries of marketing and finance MBAs: add accounting MBAs to the investigation.
If 3 independent samples of each specialty are collected (samples possibly of different sizes), we have a 1-way ANOVA situation with 3 levels.
If GPA brackets are formed, and if one samples MBAs per bracket, one from each specialty, then one has a blocked design. (Note: the 3 samples will be of equal size due to blocking.)
Randomization is not possible here: one can’t assign each student to a specialty, and one doesn’t know the GPA beforehand for matching => No causal inference.

20. Models of fixed and random effects
Fixed effects
If all possible levels of a factor are included in our analysis or the levels are chosen in a nonrandom way, we have a fixed effect ANOVA.
The conclusion of a fixed effect ANOVA applies only to the levels studied.
Random effects
If the levels included in our analysis represent a random sample of all the possible levels, we have a random-effect ANOVA.
The conclusion of the random-effect ANOVA applies to all the levels (not only those studied).

21. Models of fixed and random effects (cont.)
Fixed and random effects - examples
Fixed effects - The advertisement Example (15.1): All the levels of the marketing strategies considered were included. Inferences don’t apply to other possible strategies such as emphasizing nutritional value.
Random effects - To determine if there is a difference in the production rate of 50 machines in a large factory, four machines are randomly selected and the number of units each produces per day for 10 days is recorded.

22. 15.4 Randomized Blocks Analysis of Variance
The purpose of designing a randomized block experiment is to reduce the within-treatments variation, thus increasing the relative amount of between treatment variation.
This helps in detecting differences between the treatment means more easily.

23. Examples of Randomized Block Designs
Factor
Response
Units
Block
Varieties of Corn
Yield
Plots of Land
Adjoining plots
Blood pressure Drugs
Hypertension
Patient
Same age, sex, overall condition
Management style
Worker productivity
Amount produced by worker
Shifts

24. Randomized Blocks
Block all the observations with some commonality across treatments
Treatment 4
Treatment 3
Treatment 2
Treatment 1
Block3
Block2
Block 1

25. Randomized Blocks
Block all the observations with some commonality across treatments

26. Partitioning the total variability
The sum of square total is partitioned into three sources of variation
Treatments
Blocks
Within samples (Error)
Recall. For the independent samples design we have:
SS(Total) = SST + SSE
SS(Total) = SST + SSB + SSE
Sum of square for treatments
Sum of square for blocks
Sum of square for error

27. Sums of Squares Decomposition
= observation in ith block, jth treatment
= mean of ith block
= mean of jth treatment

28. Calculating the sums of squares
Formulas for the calculation of the sums of squares
SSB=
SST =

29. Calculating the sums of squares
Formulas for the calculation of the sums of squares
SSB=
SST =

30. Mean Squares
To perform hypothesis tests for treatments and blocks we need
Mean square for treatments
Mean square for blocks
Mean square for error

31. Test statistics for the randomized block design ANOVA
Test statistic for treatments
Test statistic for blocks
df-T: k df-B: b df-E: n-k-b+1

32. The F test rejection regions
Testing the mean responses for treatments
F > Fa,k-1,n-k-b+1
Testing the mean response for blocks
F> Fa,b-1,n-k-b+1

33. Randomized Blocks ANOVA - Example
Are there differences in the effectiveness of cholesterol reduction drugs?
To answer this question the following experiment was organized:
25 groups of men with high cholesterol were matched by age and weight. Each group consisted of 4 men.
Each person in a group received a different drug.
The cholesterol level reduction in two months was recorded.
Can we infer from the data in Xm15-02 that there are differences in mean cholesterol reduction among the four drugs?

34. Randomized Blocks ANOVA - Example
Solution
Each drug can be considered a treatment.
Each 4 records (per group) can be blocked, because they are matched by age and weight.
This procedure eliminates the variability in cholesterol reduction related to different combinations of age and weight.
This helps detect differences in the mean cholesterol reduction attributed to the different drugs.

35. Randomized Blocks ANOVA - Example
Conclusion: At 5% significance level there is sufficient evidence
to infer that the mean “cholesterol reduction” gained by at least
two drugs are different.
Treatments
Blocks
b-1
K-1
MST / MSE
MSB / MSE