2 Review Recall the idea of confounding. Suppose I want to draw inference about a certain predictor. If meaningfully different interpretations would be made depending on whether a nuisance variable is included in the model, we say that the predictor of interest is confounded with the nuisance variable. (Both will be significant – if not it is collinearity instead.) In order to draw correct conclusions, the nuisance variable must be included in the model.
3 Definitions We considered confounding for continuous predictors of interest, but it can certainly occur for categorical factors as well. We have different names for confounding depending on the situation: A (necessary) nuisance variable that is continuous is generally called a covariate. If it is another categorical factor, we usually refer to it as a blocking variable.
4 Interaction In either case, we make an assumption that the nuisance variable does not interact with the predictor of interest. If there were interaction, then we have a more complicated interaction model and BOTH variables must become variables of equal interest. This becomes either Two-Way ANOVA or Multiple Regression. For now, assume no interaction.
5 The Big Picture The nuisance variable does one of two things: Its inclusion MAY change our perspective in the sense that the pattern of differences (pairwise comparisons) for the factor of interest will change. Alternatively, its inclusion may simply be necessary to reduce the MSE so that we can actually see differences.
6 Key to Understanding View ANCOVA and Blocking as one and the same. In both situations, we are removing variation due to a nuisance variable. We are also “adjusting” the response variable for the nuisance variable – and then will compare the ADJUSTED treatment means (which may differ from the actual treatment means). Comparisons of the unadjusted means could be inaccurate.
7 Examples We now consider a couple of examples. In both examples, we will consider the effect of three standard treatments for a certain cancer. All patients are started on their respective treatment at the same time. The response variable is the lifetime of the patient in months after beginning treatment.
8 Example Dataset I
9 Example Dataset II
10 Pairwise Comparisons Data Set I Data Set II
11 Conclusions? Example I Example II
12 The missing piece? Each person is not at the same stage of disease. Some of them may have developed the cancer only recently, while others have had it for a long time. What does the picture look like if we consider this?
13 Example Dataset I
14 Example Dataset II
15 Adjusted Pairwise Comparisons Dataset I (Adjusted) Dataset I (Unadjusted)
16 Adjusted Pairwise Comparisons (2) Dataset II (Adjusted) Dataset II (Unadjusted)
17 Revised Conclusions? Example I Example II
18 Importance Failure to adjust for the covariate in either of these examples that we have seen would be a big mistake (and clearly in this case the cost would be quite high). In Example I, we’d choose what is in fact the worst treatment. In Example II, we’d fail to differentiate between the treatments when in fact one of them is clearly better than the rest.
19 ANCOVA One-Factor ANOVA with one (or more) continuous covariate(s).
20 ANCOVA Model (Cell Means) Usual assumptions / Can use factor effects instead. One additional assumption: NO INTERACTION. Could (roughly) think of “adjusting first” and modeling:
21 ANCOVA Model (2) Making the adjustment first would require knowing the true value of beta. Since we can’t know this, we essentially use the estimate of beta to make the adjustment and then compare means (SAS: LSMEANS). The adjustment effectively “levels the playing field” so that the comparison of means is made with all observations on equal footing (in the above case to X = X_bar).
22 ANCOVA Model (3) It is somewhat important to understand how SAS works (with LSMeans). Instead of using X=0 as the standard, SAS uses X-bar. So all of the means are compared at X = X-bar. This means that the adjusted means (for SAS) will be:
23 Mathematical Detail You don’t necessarily need to understand all of the mathematics behind the adjustment. (If you are interested, feel free to see me outside of class.) But it is very important to understand... WHEN and WHY an adjustment is needed. HOW to get the adjusted means in SAS and compare them. THAT using a MEANS statement is 100% WRONG any time there is a covariate involved.
24 Analysis Procedure Check assumptions as usual. Check interaction assumption by doing an “interaction plot” (more later). Check significance of variables (covariate listed first in model) using Type I SS (added in order). Marginally significant covariates should be included. Do pairwise comparisons for your factor using LSMEANS statement.
25 Details: Interaction Plot Plot the response versus the covariate with different lines representing the different levels of the factor. Judge whether the lines are mostly parallel. Can also add an interaction term to the model (costs an additional a – 1 DF) and check for its significance.
26 Example I (Code)
27 Example I (Plot)
28 Example I (Model)
29 Example I Assumption of “no interaction” seems reasonable – proceed with the ANCOVA model analysis. Might try producing these plots for the second example on your own.
30 Details: LSMeans An LSMeans statement in SAS is used to obtain and compare the “adjusted” factor level means. The syntax of the LSMeans statement is similar to the Means statement (which as we know will produce incorrect results). We can still use pairwise comparison procedures on the adjusted means (or look at contrasts of the adjusted means, etc).
31 Options in LSMeans Options come after “/” as usual. ADJUST = can be used to adjust critical values to BON, DUNNETT, SCHEFFE, or TUKEY (default is T – no adjustment) ALPHA = 0.xxx can be used to set the significance level CL calls for confidence limits for the individual means. If used in conjunction with PDIFF, it also produces confidence limits for differences.
32 Options in LSMeans (2) OUT = produces an output dataset that contains the “adjusted” means and their standard errors. This can be useful if you want to set up a contrast. PDIFF requests the p-values for pairwise comparisons (the format in which they appear is slightly different than for the means statement) STDERR requests the standard errors. TDIFF requests the t-values for pairwise differences (similar information to PDIFF)
33 Output (1)
34 Output (2)
35 Output (3) – Add CL option
36 Interpretations After Tukey adjustment is made, we don’t find significant differences when we look at pairwise comparisons. We’d still tend toward Treatment 3 – thinking that we may simply be lacking in power. (If we used LSD, we would see Treatment 3 as being better). Another experiment would be necessary to confirm this (but possibly unethical in this case – why???).
37 Other Observations Although the cell sizes were equal, the standard errors are not the same. Why? The standard errors will also involve the X’s. Recall that standard errors for mean responses increase as one gets further away from X-bar. It happened in this case that the X’s for observations that received treatment 2 were closer to X-bar. This means that the adjusted treatment mean for that group will have a smaller standard error than the other group.
38 CLG Activity #1 Question #10.1 relates to Example II Question #10.2 gives you the chance to analyze another example.
39 Randomized Complete Blocks Two-way ANOVA where one factor is important to us while the other is simply a nuisance factor.
40 Two-way ANOVA (Briefly) Interested in a combination of two factors (e.g. diet + exercise program on blood pressure). Factor Effects model used to distinguish effects: Could treat each combination of factors as a treatment and do two-way ANOVA
41 Two-way ANOVA (Briefly) The “Betas” in this model are effects, not slope estimates. Be careful not to get confused! Key to avoid confusion: If the beta is written in the model with a subscript ‘j’, then it is a factor effect. If on the other hand it does not have such a subscript and is multiplied by an ‘X’, then it is a slope parameter for the continuous variable X.
42 Interaction The term represents the potential interaction between the two factors. For now, we will still be assuming “no interaction” as we discuss blocking. We will go into interactions in greater detail in the next topic when we talk about Two-way ANOVA in general.
43 Randomized Complete Block Design RCBD is used when... We have one factor of interest in studying our response variable. We have another nuisance factor that has an effect on the response variable, but the effect is not of interest. It is important that we have some experimental control with regard to the nuisance factor.
44 Randomization If levels of nuisance variable cannot be measured / controlled, then the best we can do is to use one-way ANOVA and make sure to randomize the treatments to the experiment units. Why is such a randomization important? If the levels of the nuisance variable can be observed (but not controlled), we may be able to use stratified sampling to obtain a RCBD.
45 RCBD In a randomized complete block design, we set things up as follows: Each treatment is applied to one experimental unit within each block (or sometimes more than once if the design is replicated, but we almost always try to keep the design balanced). The treatments are randomized within the blocks.
46 Design Charts Often use a “design chart” to illustrate the experimental design. List factors in rows and columns, use “x” to represent a data point that will be collected. Helps to determine degrees of freedom for treatments, interactions, etc.
47 Design Chart Example Goal: Determine whether grade level has an effect on the length of time a teacher will stay in the profession In each of 15 major cities, one teacher from each grade level 1 through 8 was followed from the beginning of their career. This yields 120 observations; city is a blocking variable.
48 Design Chart Example
49 If 2 observations per cell…
50 Back to RCBD… In a blocking design we generally use only one replicate. So we wind up with: b blocks each consisting of a experimental units (EU’s), for a total of N = ab observations. The treatments are randomly assigned to EU’s within a block (so not totally randomized – but stratified in such a way that each block contains each treatment once).
51 Recall: Paired T Tests Blocking is essentially an extension of the paired T-test (instead of pairs, we now have blocks containing enough EU’s to apply each of the treatments within a block). Remember the point of pairing was that we improve precision by removing explainable variation from the analysis. The same thing occurs when we block – we remove variation due to the blocks first and then analyze our treatments. The similarities of blocking to ANCOVA are also quite clear. Both methods (if correctly applied) reduce the MSE and give more precision.
52 Statistical Model for RCBD Assume we have b blocks & a treatments In addition to typical assumptions, we assume that the effect due to the blocking variable is additive (i.e. there is no interaction between block and treatment). Reminder: Betas are (re)used for the block effects - they are NOT slope parameters.
53 Statistical Model for RCBD
54 Model Restrictions Common parameter restrictions are (makes the model unique and the grand mean) SAS of course uses something a little different, setting
55 ANOVA Table for RCBD SourcedfSSMSF0F0 BlockSS BLK MS BLK TreatmentSS TRT MS TRT ErrorSS ERR MS ERR TotalSS TOT MS TOT
56 Breakdown of SS When the design is balanced, Type I and Type III SS will be the same. And they will add up to the model SS. So we can obtain an “extended” ANOVA table by combining the two pieces of SAS output (ANOVA Table with model line replaced by Type I SS). There are calculation formulas in the text – I’m not going to require you to worry about them.
57 Tests Overall F test Null Hypothesis is that all ALPHAS and BETAS are zero. Rejection means something is significant, but we won’t know which factor is involved. Testing for Block / Treatment Effects Model SS can be separated into two components. For a balanced design, it won’t matter whether you are using Type I or III (they’ll be the same).
58 Tests (2) Test for “block” effect isn’t really important to us – but we can use the Type I or III SS to get an F statistic and test If “block” is even marginally significant, it should be left in the model for the reduction in MSE. Pairwise comparisons are irrelevant (why?)
59 Tests (3) Test for “treatment” is important, and we may similarly use the Type I or III SS to get an F statistic and test If the treatment tests significant then we would want to do pairwise comparisons using LSMeans. Note: It IS actually ok to use MEANS for a balanced RCBD. But it might be simpler for you to just use LSMeans all the time.
60 RCBD Multiple Comparisons Similar procedures as one-way ANOVA (Tukey, Scheffe, etc.) n is replaced by b in all formulas Degrees of freedom error is Use comparison procedure that is appropriate to what you are testing.
61 Example: Problem 18.7 (1) A study has been designed to compare body sizes of three genotypes of silkworm. Measurements include mean lengths for separately reared cocoons of heterozygous (HET), homozygous (HOM), and wild (WLD) silkworms. There were five laboratory sites and one of each type was measured at each site. VarSite #1Site #2Site #3Site #4Site #5 HET HOM WLD
62 Example: Problem 18.7 (2) Is a randomized block analysis appropriate? Need to be able to assume that the blocks (SITE) do not interact with the treatment (TYPE). Not necessarily true – would need to consult with a scientist to determine whether the different “climates” would affect the various silkworm varieties differently. If so, then we should have an interaction model, not an RCBD. For the purposes of illustration we will assume it’s true but try to check our assumption.
63 ANOVA Table (from SAS)
64 Assumptions Normality / Constant Variance OK
65 Comments Type tests as important (it turns out that it would even if we failed to block, but the blocks do remove some variation). Additivity assumption appears to be met. We proceed to analyze differences using LSMeans.