1. Inference for Regression
QTM1310/ Sharpe
Inference for Regression
In Lesson 6, we modeled relationships by fitting a straight line to a sample of ordered pairs. The Nambe Mills regression line is
Now we want to know, how useful is this model?

2. 16.1 The Population and the Sample
QTM1310/ Sharpe
16.1 The Population and the Sample
The Nambe Mills sample is based on 59 observations.
But we know observations vary from sample to sample. So we imagine a true line that summarizes the relationship between x and y for the entire population,
Where µy is the population mean of y at a given value of x.
We write µy instead of y because the regression line assumes that the means of the y values for each value of x fall exactly on the line. 2

3. 16.1 The Population and the Sample
QTM1310/ Sharpe
16.1 The Population and the Sample
For a given value x:
Most, if not all, of the y values obtained from a particular sample will not lie on the line.
The sampled y values will be distributed about µy.
We can account for the difference between ŷ and µy by adding the error residual, or ε : 3

4. 16.1 The Population and the Sample
QTM1310/ Sharpe
16.1 The Population and the Sample
Regression Inference
Collect a sample and estimate the population β’s by finding a regression line (Chapter 6):
The residuals e = y – ŷ are the sample based versions of ε.
Account for the uncertainties in β0 and β1 by making confidence intervals, as we’ve done for means and proportions. 4

5. 16.2 Assumptions and Conditions
QTM1310/ Sharpe
16.2 Assumptions and Conditions
The inference methods of Chapter 16 are based on these assumptions (check these assumptions in this order):
Linearity Assumption
Independence Assumption
Equal Variance Assumption
Normal Population Assumption 5

6. 16.2 Assumptions and Conditions
QTM1310/ Sharpe
16.2 Assumptions and Conditions
The inference methods of Chapter 16 are based on these assumptions (check these assumptions in this order):
Linearity Assumption – This condition is satisfied if the scatterplot of x and y looks straight.
2. Independence Assumption – Look for randomization in the sample or the experiment. Also check the residual plot for lack of patterns. 6

7. 16.2 Assumptions and Conditions
QTM1310/ Sharpe
16.2 Assumptions and Conditions
3. Equal Variance Assumption – Check the Equal Spread Condition, which means the variability of y should be about the same for all values of x.
4. Normal Population Assumption – Assume the errors around the idealized regression line at each value of x follow a Normal model. Check if the residuals satisfy the Nearly Normal Condition. 7

8. 16.2 Assumptions and Conditions
QTM1310/ Sharpe
16.2 Assumptions and Conditions
Summary of Assumptions and Conditions 8

9. 16.2 Assumptions and Conditions
QTM1310/ Sharpe
16.2 Assumptions and Conditions
Summary of Assumptions and Conditions
Make a scatterplot of the data to check for linearity. (Linearity Assumption)
Fit a regression and find the residuals, e, and predicted values ŷ.
Plot the residuals against time (if appropriate) and check for evidence of patterns (Independence Assumption).
Make a scatterplot of the residuals against x or the predicted values. This plot should not exhibit a “fan” or “cone” shape. (Equal Variance Assumption) 9

10. 16.2 Assumptions and Conditions
QTM1310/ Sharpe
16.2 Assumptions and Conditions
Testing the Assumptions, continued
5. Make a histogram and Normal probability plot of the residuals (Normal Population Assumption)
Data from Nambé Mills (Chapter 8) 10

11. 16.3 The Standard Error of the Slope
QTM1310/ Sharpe
16.3 The Standard Error of the Slope
For a sample, we expect b1 to be close, but not equal to the model slope β1. For similar samples, the standard error of the slope is a measure of the variability of b1 about the true slope β1. 11

12. 16.3 The Standard Error of the Slope
QTM1310/ Sharpe
16.3 The Standard Error of the Slope
Which of these scatterplots would give the more consistent regression slope estimate if we were to sample repeatedly from the underlying population?
Hint: Compare se’s. 12

13. 16.3 The Standard Error of the Slope
QTM1310/ Sharpe
16.3 The Standard Error of the Slope
Which of these scatterplots would give the more consistent regression slope estimate if we were to sample repeatedly from the underlying population?
Hint: Compare sx’s. 13

14. 16.3 The Standard Error of the Slope
QTM1310/ Sharpe
16.3 The Standard Error of the Slope
Which of these scatterplots would give the more consistent regression slope estimate if we were to sample repeatedly from the underlying population?
Hint: Compare n’s. 14

15. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope 15

16. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope
The usual null hypothesis about the slope is that it’s equal to 0. Why? A slope of zero says that y doesn’t tend to change linearly when x changes. In other words, if the slope equals zero, there is no linear association between the two variables. 16

17. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope 17

18. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope 18

19. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope
Example : Soap
A soap manufacturer tested a standard bar of soap to see how long it would last. A test subject showered with the soap each day for 15 days and recorded the weight (in grams) remaining. Conditions were met so a linear regression gave the following:
Dependent variable is: Weight
R squared = 99.5% s = 2.949
Variable Coefficient SE(Coeff) t-ratio P-value
Intercept <0.0001
Day <0.0001
What is the standard deviation of the residuals?
What is the standard error of ?
What are the hypotheses for the regression slope?
At α = 0.05, what is the conclusion? 19

20. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope
Example : Soap
A soap manufacturer tested a standard bar of soap to see how long it would last. A test subject showered with the soap each day for 15 days and recorded the weight (in grams) remaining. Conditions were met so a linear regression gave the following:
Dependent variable is: Weight
R squared = 99.5% s = 2.949
Variable Coefficient SE(Coeff) t-ratio P-value
Intercept <0.0001
Day <0.0001
What is the standard deviation of the residuals? se = What is the standard error of ? SE( ) = 20

21. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope
Example : Soap
A soap manufacturer tested a standard bar of soap to see how long it would last. A test subject showered with the soap each day for 15 days and recorded the weight (in grams) remaining. Conditions were met so a linear regression gave the following:
Dependent variable is: Weight
R squared = 99.5% s = 2.949
Variable Coefficient SE(Coeff) t-ratio P-value
Intercept <0.0001
Day <0.0001
What are the hypotheses for the
regression slope?
At α = 0.05, what is the conclusion? Since the p-value is small (<0.0001), reject the null hypothesis. There is strong evidence of a linear relationship between Weight and Day. 21

22. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope
Example : Soap
A soap manufacturer tested a standard bar of soap to see how long it would last. A test subject showered with the soap each day for 15 days and recorded the weight (in grams) remaining. Conditions were met so a linear regression gave the following:
Dependent variable is: Weight
R squared = 99.5% s = 2.949
Variable Coefficient SE(Coeff) t-ratio P-value
Intercept <0.0001
Day <0.0001
Find a 95% confidence interval for the slope?
Interpret the 95% confidence interval for the slope?
At α = 0.05, is the confidence interval consistent with the hypothesis test conclusion? 22

23. 16.4 A Test for the Regression Slope
QTM1310/ Sharpe
16.4 A Test for the Regression Slope
Example : Soap A soap manufacturer tested a standard bar of soap to see how long it would last. A test subject showered with the soap each day for 15 days and recorded the weight (in grams) remaining. Conditions were met so a linear regression gave the following:
Dependent variable is: Weight
R squared = 99.5% s = 2.949
Variable Coefficient SE(Coeff) t-ratio P-value
Intercept <0.0001
Day <0.0001
Find a 95% confidence interval for the slope?
Interpret the 95% confidence interval for the slope? We can be 95% confident that weight of soap decreases by between and 5.8 grams per day.
At α = 0.05, is the confidence interval consistent with the hypothesis test conclusion? Yes, the interval does not contain zero, so reject the null hypothesis. 23

24. 16.5 A Hypothesis Test for Correlation
QTM1310/ Sharpe
16.5 A Hypothesis Test for Correlation
What if we want to test whether the correlation between x and y is 0? 24

25. 16.6 Standard Errors for Predicted Values
QTM1310/ Sharpe
16.6 Standard Errors for Predicted Values
SE becomes larger the further xν gets from . That is, the confidence interval broadens as you move away from . (See figure at right.) 25

26. 16.6 Standard Errors for Predicted Values
QTM1310/ Sharpe
16.6 Standard Errors for Predicted Values
SE, and the confidence interval, becomes smaller with increasing n.
SE, and the confidence interval, are larger for samples with more spread around the line (when se is larger). 26

27. 16.6 Standard Errors for Predicted Values
QTM1310/ Sharpe
16.6 Standard Errors for Predicted Values
Because of the extra term , the confidence interval for individual values is broader that those for the predicted mean value. 27

28. 16.7 Using Confidence and Prediction Intervals
QTM1310/ Sharpe
16.7 Using Confidence and Prediction
Intervals
Confidence interval for a mean:
The result at 95% means
“We are 95% confident that the mean value of y is between 4.40 and 4.70 when x = 10.1.” 28

29. 16.7 Using Confidence and Prediction Intervals
QTM1310/ Sharpe
16.7 Using Confidence and Prediction
Intervals
Prediction interval for an individual value:
The result at 95% means
“We are 95% confident that a single
measurement of y will be between 3.95
and 5.15 when x = 10.1.” 29

30. 16.7 Using Confidence and Prediction Intervals
QTM1310/ Sharpe
16.7 Using Confidence and Prediction
Intervals
Example : External Hard Disks
A study of external disk drives reveals a linear relationship between the Capacity (in GB) and the Price (in $). Regression resulted in the following:
Find the predicted Price of a 1000 GB hard drive.
Find the 95% confidence interval for the mean Price of all 1000 GB hard drives.
Find the 95% prediction interval for the Price of one 1000 GB hard drive. 30

31. 16.7 Using Confidence and Prediction Intervals
QTM1310/ Sharpe
16.7 Using Confidence and Prediction
Intervals
Example : External Hard Disks
A study of external disk drives reveals a linear relationship between the Capacity (in GB) and the Price (in $). Regression resulted in the following:
Find the predicted Price of a 1000 GB hard drive. 31

32. 16.7 Using Confidence and Prediction Intervals
QTM1310/ Sharpe
16.7 Using Confidence and Prediction
Intervals
Example : External Hard Disks
A study of external disk drives reveals a linear relationship between the Capacity (in GB) and the Price (in $). Regression resulted in the following:
Find the 95% confidence interval for the mean Price of all 1000 GB hard drives. 32

33. 16.7 Using Confidence and Prediction Intervals
QTM1310/ Sharpe
16.7 Using Confidence and Prediction
Intervals
Example : External Hard Disks
A study of external disk drives reveals a linear relationship between the Capacity (in GB) and the Price (in $). Regression resulted in the following:
Find the 95% prediction interval for the Price of one 1000 GB hard drive. 33

34. Don’t fit a linear regression to data that aren’t straight.
QTM1310/ Sharpe
Don’t fit a linear regression to data that aren’t straight.
Watch out for changing spread.
Watch out for non-Normal errors. Check the histogram and the Normal probability plot.
Watch out for extrapolation. It is always dangerous to predict for x-values that lie far away from the center of the data. 34

35. Watch out for high-influence points and unusual observations.
QTM1310/ Sharpe
Watch out for high-influence points and unusual observations.
Watch out for one-tailed tests. Most software packages perform only two-tailed tests. Adjust your P-values accordingly. 35

36. QTM1310/ Sharpe
What Have We Learned?
Apply your understanding of inference for means using Student’s t to inference about regression coefficients.
Know the Assumptions and Conditions for inference about regression coefficients and how to check them, in this order:
Linearity
Independence
Equal Variance
Normality 36

37. QTM1310/ Sharpe
What Have We Learned?
Know the components of the standard error of the slope coefficient:
The standard deviation of the residuals,
The standard deviation of x,
The sample size, n 37

38. QTM1310/ Sharpe
What Have We Learned?
Be able to find and interpret the standard error of the slope.
The standard deviation of the residuals,
The standard error of the slope is the estimated standard deviation of the sampling distribution of the slope. 38

39. QTM1310/ Sharpe
What Have We Learned?
State and test the standard null hypothesis on the slope.
H0: β1 = 0. This would mean that x and y are not linearly related.
We test this null hypothesis using the t-statistic 39

40. QTM1310/ Sharpe
What Have We Learned?
Know how to use a t-test to test whether the true correlation is zero.
Construct and interpret a confidence interval for the predicted mean value corresponding to a specified value, xn.
where 40

41. QTM1310/ Sharpe
What Have We Learned?
Construct and interpret a confidence interval for an individual predicted value corresponding to a specified value, xn.
where 41

42. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
Consider the following study of the Sugar content vs. the Calorie content of breakfast cereals:
There is no obvious departure from the linearity assumption.

43. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
The histogram of residuals looks fairly normal… 43

44. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
…but the distribution shows signs of being a composite of three groups of cereal types.
The mean Calorie content may depend on some factor besides sugar content. 44

45. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
Examining the residuals of groups…
…suggests factors other than sugar content that may be important in determining Calorie content.
Puffing: replacing cereal with “air” lowers the Calorie content, even for high-sugar cereals
Fat/oil: Fats add to the Calorie content, even for low-sugar cereals
Puffed cereals (high air content per serving)
Cereals with fruits and/or nuts (high fat/oil content per serving)
All others 45

46. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
Conclusion:
It may be better to report three regressions, one for puffed cereals, one for high-fat cereals, and one for all others. 46

47. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
Example : Concert Venues
A concert production company examined it’s records and made the following scatterplot. The company places concerts in two venues, a smaller, more intimate theatre (plotted with blue circles) and a larger auditorium style venue.
Describe the relationship
between Talent Cost
and Total Revenue.
How are the results for the
two venues similar? Different? 47

48. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
Example : Concert Venues
A concert production company examined it’s records and made the following scatterplot. The company places concerts in two venues, a smaller, more intimate theatre (plotted with blue circles) and a larger auditorium style venue.
Describe the relationship
between Talent Cost
and Total Revenue.
Positive, linear, and
moderately strong. As Talent
Cost increases, Revenue
also increases. 48

49. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
Example : Concert Venues
A concert production company examined it’s records and made the following scatterplot. The company places concerts in two venues, a smaller, more intimate theatre (plotted with blue circles) and a larger auditorium style venue.
How are the results for the
two venues similar?
Both venues show an
increase of revenue with
talent cost.
Different? The larger venue has
greater variability. Revenue for
that venue is more difficult to
predict. 49

50. 17.1 Examining Residuals for Groups
QTM1310/ Sharpe
17.1 Examining Residuals for Groups
Example : Concert Venues
A concert production company examined it’s records and made the following scatterplot. The company places concerts in two venues, a smaller, more intimate theatre (plotted with blue circles) and a larger auditorium style venue.
How are the results for the
two venues different?
The larger venue has greater
variability. Revenue for that
venue is more difficult to
predict. 50

51. 17.2 Extrapolation and Prediction
QTM1310/ Sharpe
17.2 Extrapolation and Prediction
Extrapolating – predicting a y value by extending the regression model to regions outside the range of the x-values of the data. 51

52. 17.2 Extrapolation and Prediction
QTM1310/ Sharpe
17.2 Extrapolation and Prediction
Why is extrapolation dangerous?
It introduces the questionable and untested assumption that the relationship between x and y does not change. 52

53. 17.2 Extrapolation and Prediction
QTM1310/ Sharpe
17.2 Extrapolation and Prediction
Cautionary Example: Oil Prices in Constant Dollars
Model Prediction (Extrapolation):
On average, a barrel of oil will increase $7.39 per year from 1983 to 1998. 53

54. 17.2 Extrapolation and Prediction
QTM1310/ Sharpe
17.2 Extrapolation and Prediction
Cautionary Example: Oil Prices in Constant Dollars
Actual Price Behavior
Extrapolating the model to the ’80s and ’90s lead to grossly erroneous forecasts. 54