Multiple Regression Reference: Chapter 18 of Statistics for Management and Economics, 7 th Edition, Gerald Keller. 1

Earlier: Simple Linear Regression Y X + randomness Now we extend the simple linear regression model to multiple regression that allows any number of independent variables. We expect to build a model that fits the data better than the simple linear regression mode 2

Multiple regression: Two or more independent variables 3

Multiple Regression Model: Y =  0 +  1 X 1 +  2 X 2 + …+  k X k +  Y – dependent variable X 1, X 2, …, X k - k independent variables An independent variable can be function of others: Eg. X 2 =X 1 2,X 5 =X 3 X 4,.... 4

X 1 share price Y X 2 interest returnX 3 inflation Eg: 5

Motive: More ”realistic” models Better predictions Separate effects of the different variables 6

y =  0 +  1 x X y The simple linear regression model allows for one independent variable, “x” y =  0 +  1 x +  7

y =  0 +  1 x X y X2X2 1 The multiple linear regression model allows for more than one independent variable. Y =  0 +  1 x 1 +  2 x 2 +  8

y =  0 +  1 x X y X2X2 1 The multiple linear regression model allows for more than one independent variable. Y =  0 +  1 x 1 +  2 x 2 +  Note how the straight line becomes a plain, and... y =  0 +  1 x 1 +  2 x 2 9

The error  is normally distributed. The mean is equal to zero and the standard deviation is constant (    for all values of y. The errors are independent. Required conditions for the error variable 10

Earlier: Procedure for Regression Diagnostics …… ……. ….... Determine the regression equation. Check the required conditions for the errors. ……. …… If the model fits the data, use the regression equation. 11

Now Estimate  0,  1,  2,…  k with the estimates b 0, b 1, b 2, …, b k using the LS-method. Use SPSS for estimation and get the least square regression equation In practice, before trying to interpret regression coefficients, we check how well the model fits 12

–If the model assessment indicates good fit to the data, use it to interpret the coefficients and generate predictions. –Assess the model fit using statistics obtained from the sample. –Diagnose violations of required conditions. Try to remedy problems when identified. Estimating the Coefficients and Assessing the Model The procedure used to perform regression analysis: –Obtain the model coefficients and statistics using a statistical software. 13

Example: Where to locate a new motor inn? –La Quinta Motor Inns is planning an expansion. –Management wishes to predict which sites are likely to be profitable. –Several areas where predictors of profitability can be identified are: Competition Market awareness Demand generators Demographics Physical quality Estimating the Coefficients and Assessing the Model, Example 14

Profitability Competition Market awareness CustomersCommunity Physical Margin RoomsNearestOffice space College enrollment IncomeDisttwn Distance to downtown. Median household income. Distance to the nearest La Quinta inn. Number of hotels/motels rooms within 3 miles from the site. 15

Data were collected from randomly selected 100 inns that belong to La Quinta, and ran for the following suggested model: Margin =     Rooms   Nearest    Office   College +  5 Income +  6 Disttwn 16

17

Regression Analysis, SPSS Output Margin = Number Nearest Office Space Enrollment Income Distance 18

Model Assessment The model is assessed using three tools: –The standard error of estimate –The coefficient of determination –The F-test of the analysis of variance The standard error of estimates participates in building the other tools. 19

The standard deviation of the error is not known It is estimated by the Standard Error of Estimate: The magnitude of s  is judged by comparing it with Standard Error of Estimate 20

21 Similar to linear regression, coefficient of determination R 2 is interpreted

From the printout, R 2 = That is, 51.7% of the variation in operating margin is explained by the six independent variables. And 48.3% of it remains unexplained. When adjusted for degrees of freedom, Adjusted R 2 = 48.6 Coefficient of Determination 22

23 For linear relationship between dependent and independent variable(s) Earlier: Simple linear regression: Test if or not Now: Multiple regression: Test overall validity of the model

We pose the question: Is there at least one independent variable linearly related to the dependent variable? To answer the question we test the hypothesis H 0 :  0 =  1 =  2 = … =  k =0 H 1 : At least one  i is not equal to zero. If at least one  i is not equal to zero, the model has some validity. Testing the Validity of the Model 24

Model: Y =  0 +  1 X 1 +  2 X 2 + …+  6 X 6 +  where  ~N(0,  2 ) Hypothesis: H 0 :  0 =  1 =  2 = … =  k =0 H 1 : Not all  i =0 Make an overall test of the model 25

Total variation in Y =SST= SST=SSR+SSE SSR=explained variation in Y SSE=unexplained varaition in Y Bigger the SSR relative to SSE (R 2 is high) better the model Test statistic: if H 0 is true. (Degrees of freedomk=6n-k-1=93) Level of significance: Let α=

Rejection area: Reject H 0 if F obs >F crit F 0.05,6,93 ≈ 3 Observation: F obs = Conclusion: Interpretation: 27

28 We can have k number of T-tests for each co-efficient F-test in the analysis of variance combines all these T tests into one test It has lesser probability of Type 1 error than in the case of conducting multiple T tests

29