Inference for Regression Chapter 27 Inference for Regression

Objectives Review of Regression coefficients Hypothesis Tests Confidence Intervals

Inference for Regression In the last few topics we explored inference for proportions and means. The same principles used for these topics will be applied in our study of inference for regression. Our inference for regression will focus on the slope (𝛽) of the least squares regression equation. There are two forms of inference dealing with the slope, 𝛽. A t-Test for slope 𝛽 A Confidence Interval for the slope 𝛽

Two Continuous Variables Visually summarize the relationship between two continuous variables with a scatterplot Numerically, we focus on LSRL (regression) Education and Mortality Draft Order and Birthday Mortality = 1353.16 - 37.62 · Education Draft Order = 224.9 - 0.226 · Birthday

Regression Parameters When we fit a LSRL, 𝑦 =𝑎+𝑏𝑥, to a data set of two quantitative variables, we are hoping to be able to predict values of the response variable 𝑦 for given values of the explanatory variable 𝑥 . Different samples produce different estimates. The mean, 𝜇 𝑦 , of all of the possible responses has a linear relationship that represents the true regression line where 𝜇 𝑦 =𝛼+𝛽𝑥.

Regression Parameters The parameters 𝛽 and 𝛼 are estimated by a and b of the regression equation. 𝑏=𝑟 𝑠 𝑦 𝑠 𝑥 Estimate of slope  Estimate of intercept  𝑎= 𝑦 −𝑏 𝑥

Degrees of Freedom True regression line: 𝜇 𝑦 =𝛼+𝛽𝑥 The parameters 𝛼 and 𝛽 are estimated by a and b of the regression equation, 𝑦 =𝑎+𝑏𝑥, this equation having been derived from sample data. The appropriate test is a t-test for slope. Since there are two estimates in our equation, the degrees of freedom = n – 2.

Hypothesis Test for the Slope 𝛽 of a LSRL

Significance of Regression Line Does the regression line show a significant linear relationship between the two variables? If there is not a linear relationship, then we would expect zero correlation (r = 0) So the slope b should also be zero Therefore, our test for a significant relationship will focus on testing whether our slope  is significantly different from zero H0 :  = 0 versus Ha :   0  > 0  < 0

Standard Error The slope, b, varies with the sample taken. The sampling distribution model for the regression slopes is centered at the slope of the true regression line, 𝛽. We can estimate the standard error of the regression slope b: You won’t have to calculate SEb. Usually, it will be provided as part of a computer printout. s is the standard error about the line.

Standard Error The standard error about the line (s) tells us how our sample data vary from our predicted values. The standard error of the slope (SEb) tells us how our regression slope might vary with different samples. We use s in the calculation of SEb. The variability of our data affects the reliability of our slope.

Test Statistic The test statistic, tdf, is calculated in the same way as the test statistic for means and proportions.

Example of Computer Output for Regression Analysis From the printout:

Assumptions and Conditions In Chapter 8 when we fit lines to data, we needed to check only the Straight Enough Condition. Now, when we want to make inferences about the coefficients of the line, we’ll have to make more assumptions (and thus check more conditions). We need to be careful about the order in which we check conditions. If an initial assumption is not true, it makes no sense to check the later ones.

Assumptions and Conditions (cont.) Linearity Assumption: Straight Enough Condition: Check the scatterplot—the shape must be linear or we can’t use regression at all.

Assumptions and Conditions (cont.) Linearity Assumption: If the scatterplot is straight enough, we can go on to some assumptions about the errors. If not, stop here, or consider re-expressing the data to make the scatterplot more nearly linear. Check the Quantitative Data Condition. The data must be quantitative for this to make sense.

Assumptions and Conditions (cont.) Independence Assumption: Randomization Condition: the individuals are a representative sample from the population. Check the residual plot (part 1)—the residuals should appear to be randomly scattered.

Assumptions and Conditions (cont.) Equal Variance Assumption: Does The Plot Thicken? Condition: Check the residual plot (part 2)—the spread of the residuals should be uniform.

Assumptions and Conditions (cont.) Normal Population Assumption: Nearly Normal Condition: Check a histogram of the residuals. The distribution of the residuals should be unimodal and symmetric. Outlier Condition: Check for outliers.

Conditions Straight Enough Condition Quantitative Data Condition Randomization Condition Does The Plot Thicken? Condition Nearly Normal Condition Outlier Condition

Assumptions and Conditions (cont.) If all four assumptions are true, the idealized regression model would look like this: At each value of x there is a distribution of y-values that follows a Normal model, and each of these Normal models is centered on the line and has the same standard deviation.

Example: Draft Lottery Is the negative linear association we see between birthday and draft order statistically significant? H0: 𝛽 = 0 There is no linear relationship between birthday and draft order. Ha: 𝛽 ≠ 0 There is a linear relationship between birthday and draft order. p-value

Example: Draft Lottery Conclusion: P-value = 0.0001, so we reject null hypothesis and conclude that there is a statistically significant linear relationship between birthday and draft order Statistical evidence that the randomization was not done properly!

Example: Education Dataset of 78 seventh-graders: relationship between IQ and GPA Clear positive association between IQ and grade point average

Example: Education Is the positive linear association we see between GPA and IQ statistically significant? H0: 𝛽 = 0 There is no linear relationship between GPA and IQ. Ha: 𝛽 ≠ 0 There is a linear relationship between GPA and IQ. p-value

Example: Education Conclusion: P-value = 0.0001 so we reject null hypothesis and conclude that there is a statistically significant positive relationship between IQ and GPA.

Ti - 84 Xlist: Stat/Tests/LinRegTTest Ylist: Freq: 𝛽 & α: ≠0 <0 >0 RegEQ: Calculate df = n – 2

Confidence Interval for the Slope 𝛽

Confidence Interval To construct a confidence interval 𝛽, the true slope of the LSRL, we use the usual format for a confidence interval: statistic ± (critical)(standard deviation of statistic) CI for : b ± t* (SEb) The values for b and SEb are obtained from a computer printout. The t* critical value is obtained from the t-distribution with n-2 degrees of freedom.

Example: Education and Mortality

Confidence Interval for Example We have n = 60, so our critical value t* comes from a t distribution with d.f. = 58. For a 95% C.I., t* = 2.00 95 % confidence interval for slope  : -37.6 ± (2.0)(8.307) = (-54.2,-21.0) Conclusion: We are 95% confident that the true slope is between -54.2 and -21.0. Note that this interval does not contain zero!