HRP 223 – 2007 lm.ppt - Linear Models Copyright © Leland Stanford Junior University. All rights reserved. Warning: This presentation is protected by copyright law and international treaties. Unauthorized reproduction of this presentation, or any portion of it, may result in severe civil and criminal penalties and will be prosecuted to maximum extent possible under the law.

HPR lm.ppt ANOVA as a Model zAt this point, you have seen how you can build a model to describe the mean level of an outcome variable if the predictor is categorical. yYou predict the outcome at a baseline level and then add something (a constant) if you are not in the baseline group. After the prediction is made, you are left with some unexplained variability. The extra variability is assumed to be approximately normally distributed with the peak of the bell-shaped curve centered on the mean you guessed.

HPR lm.ppt ANOVA as a Model zThe model can be written as: zor to keep it simpler, baseline and 2 nd group: You can think of this as being a baseline amount (call it α) plus a change (call it β 1 ) for every one unit change in the group membership indicator for group 1. Begin to visualize the data as a bell-shaped histogram centered around the mean for group 0. You then shift the histograms for the other groups to the right or left by the amount specified as the β value.

HPR lm.ppt From the Last Slide “You can think of this as being a baseline amount (call it α) plus a change (call it β 1 ) for every one unit change in the group membership indicator (for group 1).” zInstead of a binary group membership indicator, put in a predictor variable that can take on any integer value between 0 and 10. What happens? yYou shift the bell-shaped curve up (or down if the β is negative).

HPR lm.ppt Regression! zIf you change the predictor to allow values of 0 to 10, the formula is just as simple. zConceptually, you scoot the histogram up a bit for every one unit increase in the predictor. zRemember high school?

HPR lm.ppt Continuous Predictors zIf you allow your predictor to take on any value and you are comfortable saying you are moving a bell-shaped distribution up or down, you can model the outcome with a line! zAgain, the idea is that you are just shifting your best guess at the outcome mean up by some amount (the β) for every one unit increase in the predictor.

HPR lm.ppt Mortality Rates zSay you want to look at the relationship between mortality caused by malignant melanoma and exposure to sun (as measured by the proxy of latitude). The outcome is mortality. So you will be shifting the distribution of mortality down as latitude goes North.

HPR lm.ppt Plot first of course. zA scatter plot shows the relationship between two measures on the same subject. The outcome goes on the y axis.

HPR lm.ppt A line? zThere is something like a linear relationship here. You can ask SAS to put its best guess at a line easily:

HPR lm.ppt Think about that line. zIf the best guess at the mean of the outcome does not need to be shifted up or down as the predictor changes, what will the line look like? yFLAT. yYour best guess at the outcome is just some baseline amount.

HPR lm.ppt Therefore… zThe test for the impact of a predictor in a linear model becomes a test of whether the β is close enough to 0 to call it “zero slope”.

HPR lm.ppt That Line zThe formulas to get the line are really easy. You just solve two simultaneous equations where there is a closed form solution.

HPR lm.ppt What’s going on? zIf you don’t like math, put a tack on the plot at the mean of the predictor and the mean of the outcome. Then put a ruler on the plot (touching the tack) and wiggle the ruler around until it is as close as possible to all the data points.

HPR lm.ppt Minimizing Errors

HPR lm.ppt Residuals zWhat you are doing unconsciously when you wiggle around the ruler is minimizing the errors between the line and the dots (measured up and down). These errors are called residuals.

HPR lm.ppt Guess how you measure error. zJust like every other time you have seen measurements of error, it is expressed as a variance. The model fitting process is just a process of making the line as compatible as possible with the data.

HPR lm.ppt Quality of an ANOVA Model zRemember the ANOVA model is comparing the variance across groups vs. the variance within groups. zEssentially it was saying, do you reduce the variance significantly if you use different mean lines for each subgroup of the data relative to the variance relative to a single mean?

HPR lm.ppt Quality of a Regression Model zHere you are testing to see if the variance is reduced significantly by using a sloped line rather than a flat one.

HPR lm.ppt If you like math… zThe SAS Enterprise Guide project on the class website has a data file called parts which shows how the totals accumulate for the Σ notation. Squared differences Keep a running total

HPR lm.ppt

Hypothesis Testing zThe test of the slope can be thought of as a T statistic using this formula. zFor me it is more intuitive to look at it with an ANOVA table.

HPR lm.ppt Hypothesis Testing zYou parse the sum of squares (SS) between each data point and the overall mean into two parts: yThe SS between the regression line and the overall mean yThe SS between each point and the regression line

HPR lm.ppt Partitioning the Variance

HPR lm.ppt Σ = 53,637.3 Σ = 36,464.2 Σ = 17,173.1