1. 1 1 Slide © 2008 Thomson South-Western. All Rights Reserved Chapter 15 Multiple Regression n Multiple Regression Model n Least Squares Method n Multiple Coefficient of Determination n Model Assumptions n Testing for Significance n Using the Estimated Regression Equation for Estimation and Prediction for Estimation and Prediction n Qualitative Independent Variables

2. 2 2 Slide © 2008 Thomson South-Western. All Rights Reserved The equation that describes how the dependent variable y is related to the independent variables x 1, x 2,... x p and an error term is: The equation that describes how the dependent variable y is related to the independent variables x 1, x 2,... x p and an error term is: Multiple Regression Model y =  0 +  1 x 1 +  2 x 2 +... +  p x p +  where:  0,  1,  2,...,  p are the parameters, and  is a random variable called the error term n Multiple Regression Model

3. 3 3 Slide © 2008 Thomson South-Western. All Rights Reserved The equation that describes how the mean value of y is related to x 1, x 2,... x p is: The equation that describes how the mean value of y is related to x 1, x 2,... x p is: Multiple Regression Equation E ( y ) =  0 +  1 x 1 +  2 x 2 +... +  p x p n Multiple Regression Equation

4. 4 4 Slide © 2008 Thomson South-Western. All Rights Reserved A simple random sample is used to compute sample statistics b 0, b 1, b 2,..., b p that are used as the point estimators of the parameters  0,  1,  2,...,  p. Estimated Multiple Regression Equation ^ y = b 0 + b 1 x 1 + b 2 x 2 +... + b p x p Estimated Multiple Regression Equation Estimated Multiple Regression Equation

5. 5 5 Slide © 2008 Thomson South-Western. All Rights Reserved Estimation Process Multiple Regression Model y =  0 +  1 x 1 +  2 x 2 +...+  p x p +  Multiple Regression Equation E ( y ) =  0 +  1 x 1 +  2 x 2 +...+  p x p Unknown parameters are  0,  1,  2,...,  p Sample Data: x 1 x 2... x p y.... Estimated Multiple Regression Equation Sample statistics are b 0, b 1, b 2,..., b p b 0, b 1, b 2,..., b p b 0, b 1, b 2,..., b p provide estimates of  0,  1,  2,...,  p

6. 6 6 Slide © 2008 Thomson South-Western. All Rights Reserved Least Squares Method n Least Squares Criterion n Computation of Coefficient Values The formulas for the regression coefficients The formulas for the regression coefficients b 0, b 1, b 2,... b p involve the use of matrix algebra. We will rely on computer software packages to perform the calculations.

7. 7 7 Slide © 2008 Thomson South-Western. All Rights Reserved Interpreting the Coefficients In multiple regression analysis, we interpret each In multiple regression analysis, we interpret each regression coefficient as follows: regression coefficient as follows: b i represents an estimate of the change in y b i represents an estimate of the change in y corresponding to a 1-unit increase in x i when all corresponding to a 1-unit increase in x i when all other independent variables are held constant. other independent variables are held constant.

8. 8 8 Slide © 2008 Thomson South-Western. All Rights Reserved Multiple Coefficient of Determination n Relationship Among SST, SSR, SSE where: SST = total sum of squares SST = total sum of squares SSR = sum of squares due to regression SSR = sum of squares due to regression SSE = sum of squares due to error SSE = sum of squares due to error SST = SSR + SSE = +

9. 9 9 Slide © 2008 Thomson South-Western. All Rights Reserved The variance of , denoted by  2, is the same for all The variance of , denoted by  2, is the same for all values of the independent variables. values of the independent variables. The variance of , denoted by  2, is the same for all The variance of , denoted by  2, is the same for all values of the independent variables. values of the independent variables. The error  is a normally distributed random variable The error  is a normally distributed random variable reflecting the deviation between the y value and the reflecting the deviation between the y value and the expected value of y given by  0 +  1 x 1 +  2 x 2 +.. +  p x p. expected value of y given by  0 +  1 x 1 +  2 x 2 +.. +  p x p. The error  is a normally distributed random variable The error  is a normally distributed random variable reflecting the deviation between the y value and the reflecting the deviation between the y value and the expected value of y given by  0 +  1 x 1 +  2 x 2 +.. +  p x p. expected value of y given by  0 +  1 x 1 +  2 x 2 +.. +  p x p. Assumptions About the Error Term  The error  is a random variable with mean of zero. The error  is a random variable with mean of zero. The values of  are independent. The values of  are independent.

10. 10 Slide © 2008 Thomson South-Western. All Rights Reserved In simple linear regression, the F and t tests provide In simple linear regression, the F and t tests provide the same conclusion. the same conclusion. In simple linear regression, the F and t tests provide In simple linear regression, the F and t tests provide the same conclusion. the same conclusion. Testing for Significance In multiple regression, the F and t tests have different In multiple regression, the F and t tests have different purposes. purposes. In multiple regression, the F and t tests have different In multiple regression, the F and t tests have different purposes. purposes.

11. 11 Slide © 2008 Thomson South-Western. All Rights Reserved Testing for Significance: F Test The F test is referred to as the test for overall The F test is referred to as the test for overall significance. significance. The F test is referred to as the test for overall The F test is referred to as the test for overall significance. significance. The F test is used to determine whether a significant The F test is used to determine whether a significant relationship exists between the dependent variable relationship exists between the dependent variable and the set of all the independent variables. and the set of all the independent variables. The F test is used to determine whether a significant The F test is used to determine whether a significant relationship exists between the dependent variable relationship exists between the dependent variable and the set of all the independent variables. and the set of all the independent variables.

12. 12 Slide © 2008 Thomson South-Western. All Rights Reserved A separate t test is conducted for each of the A separate t test is conducted for each of the independent variables in the model. independent variables in the model. A separate t test is conducted for each of the A separate t test is conducted for each of the independent variables in the model. independent variables in the model. If the F test shows an overall significance, the t test is If the F test shows an overall significance, the t test is used to determine whether each of the individual used to determine whether each of the individual independent variables is significant. independent variables is significant. If the F test shows an overall significance, the t test is If the F test shows an overall significance, the t test is used to determine whether each of the individual used to determine whether each of the individual independent variables is significant. independent variables is significant. Testing for Significance: t Test We refer to each of these t tests as a test for individual We refer to each of these t tests as a test for individual significance. significance. We refer to each of these t tests as a test for individual We refer to each of these t tests as a test for individual significance. significance.

13. 13 Slide © 2008 Thomson South-Western. All Rights Reserved Testing for Significance: F Test Hypotheses Rejection Rule Test Statistics H 0 :  1 =  2 =... =  p = 0 H 0 :  1 =  2 =... =  p = 0 H a : One or more of the parameters H a : One or more of the parameters is not equal to zero. is not equal to zero. F = MSR/MSE Reject H 0 if p -value F   where F  is based on an F distribution with p d.f. in the numerator and n - p - 1 d.f. in the denominator.

14. 14 Slide © 2008 Thomson South-Western. All Rights Reserved Testing for Significance: t Test Hypotheses Rejection Rule Test Statistics Reject H 0 if p -value <  or if t t   where t  is based on a t distribution with n - p - 1 degrees of freedom.

15. 15 Slide © 2008 Thomson South-Western. All Rights Reserved Testing for Significance: Multicollinearity The term multicollinearity refers to the correlation The term multicollinearity refers to the correlation among the independent variables. among the independent variables. The term multicollinearity refers to the correlation The term multicollinearity refers to the correlation among the independent variables. among the independent variables. When the independent variables are highly correlated When the independent variables are highly correlated (say, | r | >.7), it is not possible to determine the (say, | r | >.7), it is not possible to determine the separate effect of any particular independent variable separate effect of any particular independent variable on the dependent variable. on the dependent variable. When the independent variables are highly correlated When the independent variables are highly correlated (say, | r | >.7), it is not possible to determine the (say, | r | >.7), it is not possible to determine the separate effect of any particular independent variable separate effect of any particular independent variable on the dependent variable. on the dependent variable.

16. 16 Slide © 2008 Thomson South-Western. All Rights Reserved Testing for Significance: Multicollinearity Every attempt should be made to avoid including Every attempt should be made to avoid including independent variables that are highly correlated. independent variables that are highly correlated. Every attempt should be made to avoid including Every attempt should be made to avoid including independent variables that are highly correlated. independent variables that are highly correlated. If the estimated regression equation is to be used only If the estimated regression equation is to be used only for predictive purposes, multicollinearity is usually for predictive purposes, multicollinearity is usually not a serious problem. not a serious problem. If the estimated regression equation is to be used only If the estimated regression equation is to be used only for predictive purposes, multicollinearity is usually for predictive purposes, multicollinearity is usually not a serious problem. not a serious problem.

17. 17 Slide © 2008 Thomson South-Western. All Rights Reserved Using the Estimated Regression Equation for Estimation and Prediction The procedures for estimating the mean value of y The procedures for estimating the mean value of y and predicting an individual value of y in multiple and predicting an individual value of y in multiple regression are similar to those in simple regression. regression are similar to those in simple regression. The procedures for estimating the mean value of y The procedures for estimating the mean value of y and predicting an individual value of y in multiple and predicting an individual value of y in multiple regression are similar to those in simple regression. regression are similar to those in simple regression. We substitute the given values of x 1, x 2,..., x p into We substitute the given values of x 1, x 2,..., x p into the estimated regression equation and use the the estimated regression equation and use the corresponding value of y as the point estimate. corresponding value of y as the point estimate. We substitute the given values of x 1, x 2,..., x p into We substitute the given values of x 1, x 2,..., x p into the estimated regression equation and use the the estimated regression equation and use the corresponding value of y as the point estimate. corresponding value of y as the point estimate.

18. 18 Slide © 2008 Thomson South-Western. All Rights Reserved In many situations we must work with qualitative In many situations we must work with qualitative independent variables such as gender (male, female), independent variables such as gender (male, female), method of payment (cash, check, credit card), etc. method of payment (cash, check, credit card), etc. In many situations we must work with qualitative In many situations we must work with qualitative independent variables such as gender (male, female), independent variables such as gender (male, female), method of payment (cash, check, credit card), etc. method of payment (cash, check, credit card), etc. For example, x 2 might represent gender where x 2 = 0 For example, x 2 might represent gender where x 2 = 0 indicates male and x 2 = 1 indicates female. indicates male and x 2 = 1 indicates female. For example, x 2 might represent gender where x 2 = 0 For example, x 2 might represent gender where x 2 = 0 indicates male and x 2 = 1 indicates female. indicates male and x 2 = 1 indicates female. Qualitative Independent Variables In this case, x 2 is called a dummy or indicator variable. In this case, x 2 is called a dummy or indicator variable.