1. Chapter 8 Linear regression
Math2200

2. Scatterplot
One double Whopper (a kind of sandwich of Burger King) contains 53 grams of protein, 65 grams of fat and 1020 calories. So two double would contain enough calories for a day.
fat versus protein for 30 items on the Burger King menu

3. The linear model Parameters Model is NOT perfect! Predicted value
Intercept
Slope
Model is NOT perfect!
Predicted value
Residual = = Observed – predicted
Overestimate when residual<0
Underestimate when residual>0

4. The linear model (cont.)
We write our model as
This model says that our predictions from our model follow a straight line.
If the model is a good one, the data values will scatter closely around it.

5. How did I get the line? “Best fit” line Minimize residuals overall
The line of best fit is the line for which the sum of the squared residuals is smallest.
Least squares, that is to minimize

6. How did I get the line? (cont.)
The regression line
Slope: in units of y per unit of x
Intercept: in units of y

7. TI-83 Enter data as lists first Press STAT
Then move the cursor to CALC
Press 4 (LinReg(ax+b)) or Press 8 (LinReg(a+bx))
Then put the list names for which you want to do regression, e.g., L1, L2
Press ENTER
To see
Set DIAGNOSTICS ON
2ND + 0 (CATALOG), move the cursor down to DiagnosticsOn
Press ENTER (You will see ‘DONE’)
Now repeat the above operations for linear regression, you will see the correlation coefficient and

8. TI-83 How to make the residual plot? Same as making a scatterplot
Make the XLIST as the explanatory variable
Make the YLIST as ‘RESID’

9. Interpreting the regression line
Slope:
Increasing 1 unit in x  increasing units in y
Intercept: predicted value of y when x=0
Predicted value at x

10. Fat Versus Protein: An Example
The regression line for the Burger King data fits the data well:
The equation is
The predicted fat content for a BK Broiler chicken sandwich is
(30) = 35.9 grams of fat.

11. Correlation and the Line
Moving one standard deviation away from the mean in x moves us r standard deviations away from the mean in y.

12. How Big Can Predicted Values Get?
r cannot be bigger than 1 (in absolute value), so each predicted y tends to be closer to its mean (in standard deviations) than its corresponding x was.
This property of the linear model is called regression to the mean; the line is called the regression line.

13. Sir Francis Galton

14. Residuals Revisited
The residuals are the part of the data that hasn’t been modeled.
Data = Model + Residual
or (equivalently)
Residual = Data – Model
Or, in symbols,

15. Residuals Revisited (cont.)
When a regression model is appropriate, nothing interesting should be left behind.
Residual plot should have no pattern, no bend, no outlier.
Residual against x
Residual against predicted value
The spread of the residual plot should be the same throughout.

16. Residuals Revisited (cont.)
If the residuals show no interesting pattern in the residual plot, we use standard deviation of the residuals to measure how much the points spread around the regression line. The standard deviation of residuals is given by
We need the Equal Variance Assumption for the standard deviation of residuals. The associated condition to check is the Does the Plot Thicken? Condition.

17. Residual plot

18. How well does the linear model fit?
The variation in the residuals is the key to assessing how well the model fits.
Total fat (y): sd =16.4g
Residual: sd = 9.2g
less variation
How much of the variation
is accounted for by the model?
How much is left in the residuals?

19. Variation
The squared correlation gives the fraction of the data’s variation explained by the model.
We can view as the percentage of variability in y that is NOT explained by the regression line, or the variability that has been left in the residuals
For the BK model, r2 = = 0.69, so 31% of the variability in total fat has been left in the residuals.

20. R2—The Variation Accounted For
0: no variance explained
1: all variance explained by the model
How big should R2 be to conclude the model fit the data well?
R2 is always between 0% and 100%. What makes a “good” R2 value depends on the kind of data you are analyzing and on what you want to do with it.

21. Check the following conditions
The two variables are both quantitative
The relationship is linear (straight enough)
Scatterplot
Residual plot
No outliers: Are there very large residuals?
Equal variance: all residuals should share the same spread
Residual plot: Does the Plot Thicken?

22. Summary Whether the linear model is appropriate?
Residual plot
How well does the model fit? R2

23. Reality Check: Is the Regression Reasonable?
Statistics don’t come out of nowhere. They are based on data.
The results of a statistical analysis should reinforce your common sense, not fly in its face.
If the results are surprising, then either you’ve learned something new about the world or your analysis is wrong.
When you perform a regression, think about the coefficients and ask yourself whether they make sense.

24. What Can Go Wrong?
Don’t fit a straight line to a nonlinear relationship.
Beware extraordinary points (y-values that stand off from the linear pattern or extreme x-values).
Don’t extrapolate beyond the data—the linear model may no longer hold outside of the range of the data.
Don’t infer that x causes y just because there is a good linear model for their relationship—association is not causation.
Don’t choose a model based on R2 alone.

25. What have we learned?
When the relationship between two quantitative variables is fairly straight, a linear model can help summarize that relationship.
The regression line doesn’t pass through all the points, but it is the best compromise in the sense that it has the smallest sum of squared residuals.

26. What have we learned? (cont.)
The correlation tells us several things about the regression:
The slope of the line is based on the correlation, adjusted for the units of x and y.
For each SD in x that we are away from the x mean, we expect to be r SDs in y away from the y mean.
Since r is always between -1 and +1, each predicted y is fewer SDs away from its mean than the corresponding x was (regression to the mean).
R2 gives us the fraction of the response accounted for by the regression model.

27. What have we learned? (cont.)
The residuals also reveal how well the model works.
If a plot of the residuals against predicted values shows a pattern, we should re-examine the data to see why.
The standard deviation of the residuals quantifies the amount of scatter around the line.

28. What have we learned? (cont.)
The linear model makes no sense unless the Linear Relationship Assumption is satisfied.
Also, we need to check the Straight Enough Condition and Outlier Condition with a scatterplot.
For the standard deviation of the residuals, we must make the Equal Variance Assumption. We check it by looking at both the original scatterplot and the residual plot for Does the Plot Thicken? Condition.

29. Summary for Chapters 7 and 8
How to read a scatter plot?
Direction
Form
Strength
Correlation coefficient
When can you use it?
How to calculate it?
How to interpret it?
Linear regression
How to make predictions?
How to read residual plot?