Topic 12 – Further Topics in ANOVA Unequal Cell Sizes (Chapter 20)

Overview We’ll start with the Learning Activity. More practice in interpreting ANOVA results; and a baby-step into 3-way ANOVA. An illustration of the problems that an unbalanced design will cause. We’ll then continue with a discussion of unbalanced designs (Chapter 20)

Collaborative Learning Activity Take your time going through this. Ask questions as needed!

Analyze the design elements. Question 1 Analyze the design elements.

Design Chart Unequal Cell Sizes – but there is SOME balance achieved Single Factor Analyses will be balanced. Gender*Age = 6 observations per cell Time*Age = 6 observations per cell Gender*Time = Unbalanced

Question 2 Analyze Age*Time

Interaction Plot (ignoring gender)

Interpretations No interaction is evident between age and time Seems middle age group gets generally higher offers. Seems offers during the week are generally higher than on the weekend (this effect is not as big as the age effect)

Main Effects Plots

ANOVA Type I vs Type III?

LSMeans #3 (middle aged, weekday) is the highest Using Tukey comparisons it is significantly higher than all others. “Slicing” will show the same things that we guessed from the plots.

LSMeans (sliced)

Slicing of LSMeans Sums of Squares add to??? DF add to??? Effect of slicing is to look at differences for one of the two factors at a specific level of the other factor. Interpretations???

Question 3 Analyze Age*Gender

Interaction Plot (ignoring Time)

Interpretations Small interaction is seen; might be described as follows: There is still a clear main effect: Middle aged get higher offers in general There seem to be no gender differences for middle aged or young. For elderly, women may be getting lower offers than men.

LSMeans (sliced comparisons)

ANOVA / LSMeans Only age differences show up in the ANOVA. “Sliced” LSMeans comparisons do pick up gender difference within elderly Note: Type I error rate is uncontrolled. But on the other hand sample sizes are also fairly small. Conclusions?

Question 4 Analyze Time*Gender

Interaction Plot (ignoring age)

Interpretations Seems to be a clear interaction: For men, there is not much difference in the offer between weekday/weekend. Women should go on the weekdays, where it seems they average about $400 more. Interestingly, significance is not seen in the ANOVA table, but is seen in the ‘sliced’ LSMeans output. Remember Type I Error is uncontrolled.

ANOVA Table Why are Type I / Type III SS different here?

Sliced LSMeans

Conclusions This is an intriguing example, because the ANOVA output would lead you to believe there is a small time effect, but no gender effect. Looking at the interaction plot presents a completely different picture (and likely a more accurate one). Let’s reconsider that, showing the sample sizes.

Interaction Plot (ignoring age)

Confounding This picture illustrates how the effects of gender and time will be confounded. Suppose that women do get lower offers than men in general. Then because the women received more weekend offers (and men more offers on weekdays), the average offer on the weekend will by default be lower than the weekday. Simple example: Suppose men get $2 and women get $1. Then with the sample sizes, the weekday average will be 30/18 while the weekend average will be only 24/18.

3-way ANOVA Is Gender Important? Questions 5 & 6 3-way ANOVA Is Gender Important?

Modeling Removing unimportant terms (starting at the interaction level) seems like a reasonable way to go. Use Type III SS to do this since cell sizes are not the same. The procedure leads to a model containing only Age and Time; suggesting that gender is unimportant. But we know this may not be accurate since gender/time are confounded.

Confounding What exactly does it mean to say that the time/gender effects are confounded. The biggest thing that it means is that the analysis we just did is inappropriate since... The time effect may have been seen because more women went on the weekend. It may well be a gender effect that is disguised as a time effect due to the unbalanced design. Due to the lack of balance – we were forced to use Type III SS which (due to collinearity / confounding may not tell the whole story).

Importance of Gender? Probably! Direct algorithmic analysis suggests both time and age are important, while gender is not. But due to confounding, that wasn’t really appropriate. The plot for time*gender indicates what is probably the real story (due to small sample sizes it is hard to get significance). With a balanced design – we would be much better off. The effects would not be confounded, and we could therefore see an accurate picture.

Importance of Gender? (2) Differing sample sizes means that Estimates for women on weekdays, and men on weekends, will have larger standard errors. This will reduce our power to detect differences, and the effects will “overlap” to some extent because of the unequal sample sizes. When we looked at the gender*time interaction, the plot suggested there was an important one. Further studies should be conducted to determine if this is the case.

Unbalanced Two-Way ANOVA Unequal Cell Sizes (Chapter 20 – skim only)

Differing Cell Sizes Encountered for a variety of reasons including: Convenience – usually if we have an observational study, we have very little control over the cell sizes. Cost Effectiveness – sometimes the cost of samples is different, and we may use larger sample sizes when the cost is less. Accidently – In experimental studies, you may start with a balanced design, but lose that balance if some problem occurs.

Differing Cell Sizes (2) What changes? Loss of balance brings “intercorrelation” among the predictors. Type I and III SS will be different; typically Type III SS should be used for testing but as we have seen even that is not perfect! Standard errors for cell means and for multiple comparisons will be different (they depend on the cell size). For the same reason, confidence intervals will have different widths.

Example Examine the effects of gender (A) and anxiety level (B) on a toxin level in the bloodstream. Three categories of anxiety (Severe, Moderate, and Mild). We categorize people on this basis after they are in the study (it is an observational factor). For cost effectiveness, we wouldn’t want to throw away data just to keep a balanced design.

Data

Interaction Plot

Interpretation Effect seems to be greater if anxiety is more severe. This is an interaction of the “enhancement type”. The effect of anxiety level on toxin levels is greater for women than it is for men. Remember, we aren’t saying anything about significance here – we’ll do that when we look at the ANOVA.

ANOVA Output

Type I / III SS

Differences in Type I / III SS The more unbalanced the design, the further apart these may be. There are actually four types of SS: I – Sequential II – Added Last (Observation) III – Added Last (Cell) IV – Added Last (Empty Cells)

Type I SS Sequential Sums of Squares; Most appropriate for equal cell sizes. SS(A), SS(B|A), SS(A*B|A,B) Each observation is weighted equally. So the net result for an unbalanced design is that some treatments will be considered with greater weight than others.

Type II SS Variable Added Last SS; Generally only used for regression because again each observation is weighted equally. SS(A|B,A*B), SS(B|A,A*B), SS(A*B|A,B)

Type III SS Variable Added Last SS, appropriate for unequal cell sizes. Type III SS adjusts for the fact that cell sizes are different. Each cell is weighted equally, with the result that treatments are weighted equally. This means that observations in “smaller” cells will carry more weight. SS(A|B,A*B), SS(B|A,A*B), SS(A*B|A,B)

Type IV SS Variable Added Last SS and similar to Type III SS but further allows for the possibility of empty cells. It is only necessary to use these if there are empty cells (which hopefully there won’t be if you’ve designed the experiment well). SS(A|B,A*B), SS(B|A,A*B), SS(A*B|A,B)

General Strategy Remember that Type I SS and Type III SS examine different null hypotheses. Type III SS are preferred when sample sizes are not equal, but can be somewhat misleading if sample sizes differ greatly. Type IV SS are appropriate if there are empty cells. Can obtain Type IV SS if necessary by using /ss4 in MODEL statement

Example (continued) The interaction is unimportant, nor is there an apparent large effect of gender. Now look at comparing different levels of anxiety; should not ‘change’ models at this point, so just average over gender (LSMeans).

LSMeans Must use LSMeans to adjust all means to the same “average level” of gender.

Comparisons Mild group has significantly lower toxin levels than the moderate and severe groups

Confidence Intervals Could get CI’s for means and/or differences if you wanted them. They will be of different widths – why? It will be harder to detect differences for groups with fewer observations.

Questions?

Upcoming in Topic 13... Random Effects (parts of chapters 17 & 19 that were previously skipped)