Chapter 15 Multiple Regression

Regression Multiple Regression Model y =  0 +  1 x 1 +  2 x 2 + … +  p x p +  Multiple Regression Equation y =  0 +  1 x 1 +  2 x 2 + … +  p x p Estimated Multiple Regression Equation

Car Data MPGWeightYearCylinders Continuing on for 397 observations

Multiple Regression, Example CoefficientsStandard Errort Stat Intercept Weight R Square0.687 CoefficientsStandard Errort Stat Intercept Weight Year R Square0.807

Multiple Regression, Example CoefficientsStandard Errort Stat Intercept Weight Year Cylinders R Square0.807 Predicted MPG for car weighing 4000 lbs built in 1980 with 6 cylinders: (4000)+.76(80)-.0741(6) = =19.88

SST = SSR + SSE Sums of Squares

Multiple Coefficient of Determination The share of the variation explained by the estimated model. R 2 = SSR/SST Multiple Correlation Coefficient The correlation coefficient of the actual and predicted values

Adjusted Multiple Coefficient of Determination Regression Statistics Multiple R0.898 R Square0.807 Adjusted R Square0.805 Standard Error3.44 Observations397

F Test for Overall Significance H 0 :  1 =  2 =... =  p = 0 H a : One or more of the parameters is not equal to zero Reject H 0 if: F > F  Or Reject H 0 if: p-value <  F = MSR/MSE

ANOVA Table for Multiple Regression Model SourceSum of Squares Degrees of Freedom Mean SquaresF RegressionSSRpMSR = SSR/pF=MSR/MSE ErrorSSEn-p-1MSE = SSE/(n-p-1) TotalSSTn-1

ANOVA Example ANOVA dfSSMSF Significance F Regression E-140 Residual Total

t Test for Coefficients H 0 :  1 = 0 H a :  1 ≠ 0 Reject H 0 if: t t  Or if: p <  t = b 1 /s b1 With a t distribution of n-p-1 df

t Test Example CoefficientsStandard Errort StatP-value Intercept Weight E-37 Year E-41 Cylinders

Multicollinearity When two or more independent variables are highly correlated. When multicollinearity is severe the estimated values of coefficients will be unreliable.

Multicollinearity Two guidelines for identifying multicollinearity: If the absolute value of the correlation coefficient for two independent variables exceeds 0.7 If the correlation coefficient for an independent variable and some other independent variable is greater than the correlation with that variable and the dependent variable

Multicollinearity MPGWeightYearCylinders MPG1 Weight Year Cylinders Table of correlation coefficients:

Multicollinearity CoefficientsStandard Errort Stat Intercept Weight Year Cylinders R Square0.807 CoefficientsStandard Errort Stat Intercept Year Cylinders R Square 0.708

Qualitative Variables and Regression Quantitative variable – A variable that can be measured numerically (interval or ratio scale of measurement) Qualitative variable – A variable where labels or names are used to identify some attribute (nominal or ordinal scale of measurement)

Qualitative Variables and Regression The effect of a quantitative variable can be estimated using a dummy variable. A dummy variable can equal 0 or 1, it creates different y intercepts for groups with different attributes.

Qualitative Variables and Regression Assume we estimate a regression model for the number of sick days an employee takes per year. A dummy variable is included that equals 1 if the individual smokes and 0 if they do not. Age is also included in the model.

Qualitative Variables and Regression Estimated model: Sick days taken = -1 +(3)Smoker + (.1)Age Sick DaysSmokerAge Example of how data would be coded:

Dummy Variables Sick days taken = -1 +(3)Smoker + (.1)Age What is the y-intercept for nonsmokers? What is the y-intercept for smokers? 2 What is the predicted number of sick days for a 40-year-old smoker? 6 What is the average difference in the number of sick days taken by smokers and nonsmokers? 3

Dummy Variables If an attribute has three or more possible values you must include k-1 dummy variables in the model, where k is the number of possible values.

Dummy Variables Suppose we have three job classifications: manager, operator, and secretary Operator dummy equals 1 if the person is an operator, 0 otherwise Secretary dummy equals 1 if the person is an secretary, 0 otherwise Manager is the omitted group (choice of omitted group will not alter the predicted values)

Dummy Variables Sick days taken = -1 +(1)Operator + 1.5(Secretary) + (.1)Age What are the y-intercepts for each job classification? Managers=-1, Operators=0, Secretaries=0.5 What is the predicted number of sick days for a 40- year-old secretary? 4.5 What is the average difference in the number of sick days taken by operators and secretaries? 0.5

Dummy Variables In some cases there will be multiple sets of dummy variables, such as: Sick days taken = -1 +(3)Smoker + (1)Operator + 1.5(Secretary) + (.1)Age Note that there are now 6 different intercepts: Nonsmoker, Manager: -1 (omitted group) Smoker, Manager: 2 Nonsmoker, Operator: 0 Smoker, Operator: 3 Nonsmoker, Secretary: 0.5 Smoker, Secretary: 3.5

Dummy Variables Note that when dummy variables are used we are assuming that the coefficients of the other variables are the same for all groups. In this example the increase in sick days used from aging a year is equal to 0.1 for all of the groups. If there is reason to believe the effect of an independent variable differs by group, you may want to estimate separate equations for each group.

Nonlinear Relationships Nonlinear relationships can be modeled by including a variable that is a nonlinear function of an independent variable. For example it is usually assumed that health care expenditures increase at an increasing rate as people age.

Nonlinear Relationships In that case you might try including age squared into the model: Health expend = (5)Age + (.5)AgeSQ AgeHealth Expend

Nonlinear Relationships If the dependent variable increases at a decreasing rate as the independent variable rises you might want to include the square root of the independent variable. If you are unsure of the nature of the relationship you can use dummy variables for different ranges of values of the independent variable.

Non-continuous Relationships If the relationship between the dependent variable and an independent variable is non-continuous a slope dummy variable can be used to estimate two sets of coefficients for the independent variable. For example, if natural gas usage is not affected by temperature when the temperature rises above 60 degrees, we could have: Gas usage = b0 + b 1 (GT60) + b 2 (Temp) + b 2 (GT60)(Temp)

Non-continuous Relationships Note that at temperatures above 60 degrees the net effect of a 1 degree increase in temperature on gas usage is ( ) Coefficients Standard Errort StatP-value Intercept E-18 GT Temp E-13 (GT60)(Temp)