# Multiple Regression Dummy Variables Multicollinearity

## Presentation on theme: "Multiple Regression Dummy Variables Multicollinearity"— Presentation transcript:

Multiple Regression Dummy Variables Multicollinearity
Interaction Effects Heteroscedasticity

Lecture Objectives You should be able to :
Convert categorical variables into dummies. Identify and eliminate Multicollinearity. Use interaction terms and interpret their coefficients. Identify heteroscedasticity.

I. Using Categorical Data: Dummy Variables
X1 X2 Accidents per 10000 Car Obs Drivers Age Color 1 89 17 Red 2 70 Black 3 75 Blue 4 85 18 5 74 6 76 7 90 19 8 78 9 10 80 20 Consider insurance company data on accidents and their relationship to age of driver and the color of car driven. See spreadsheet for complete data.

Coding a Categorical Variable
Original Coding Alternate Coding Y X1 X2 Accidents per 10000 Car Drivers Age Color 89 17 1 3 70 2 75 85 18 74 76 90 19 78 80 20 This is the incorrect way. Output from the two ways of coding give inconsistent results.

Original Coding: Partial Output and Forecasts
SUMMARYOUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 27 RESIDUAL OUTPUT Observation Predicted Accidents Residuals 1 0.6444 2 3 4 0.0389 5 6 1.0389 7 8.4333 8 1.4333 9 10 1.8278 Coefficients Intercept Age Color

Modified Coding: Partial Output and Forecasts
SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 27 RESIDUAL OUTPUT Observation Predicted Accidents Residuals 1 2 3 4 5 0.7056 6 7 6.7667 8 8.1000 9 10 0.1611 Coefficients Intercept Age Color 6.6667

Coding with Dummies Original Dummies Alternately Coded Dummies Y X1 X2 X3 Accidents per 10000 Drivers Age D1 Red D2 Black D1 Black D2 Blue 89 17 1 70 75 85 18 74 76 90 19 78 80 20 This is the correct way. Output from either way of coding gives the same forecasts.

Regression Statistics
Original Dummy Coding SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 27 RESIDUAL OUTPUT Observation Predicted Accidents Residuals 1 2 3 4 5 6 7 5.6556 8 6.9889 9 10 Coefficients Intercept Age D1 Red D2 Black

Regression Statistics
Modified Dummy Coding SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 27 RESIDUAL OUTPUT Observation Predicted Accidents Residuals 1 2 3 4 5 6 7 5.6556 8 6.9889 9 10 Coefficients Intercept Age D1 Black D2 Blue

II. Multicollinearity We wish to forecast the height of a person based on the length of his/her feet. Consider data as shown: Height Right Left 77.31 11.59 11.54 67.58 9.57 9.63 70.40 8.97 8.98 64.84 9.39 9.46 77.03 12.05 12.03 79.66 11.39 11.41 72.37 10.55 10.61 73.18 10.31 10.33 77.60 11.81 71.40 9.92 9.88

Regression with Right Foot
SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 105 As right foot length increases by an inch, height increases on average by 3.99 inches. Coefficients Intercept Right

Regression with Left Foot
SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 105 As left foot length increases by an inch, height increases on average by 3.99 inches. Coefficients Intercept Left

Regression Statistics
Regression with Both SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 105 As right foot length increases by an inch, height increases on average by 8.52 inches (assuming left foot is constant!) while lengthening of the left foot makes a person shorter by 4.55 inches!! Coefficients Intercept Right Left

The Reason? Multicollinearity.
Height Right Left Height (y) 1.0000 Right (X1) 0.9031 Left (X2) 0.9003 0.9990 While both feet (Xs) are correlated with height (y), they are also highly correlated with each other (0.999). In other words, the second foot adds no extra information to the prediction of y. One of the two Xs is sufficient.

III. Interaction Effects
Scores on test of reflexes Y X1 X2 Obs Score Age Gender 1 80 25 2 82 28 3 75 32 4 70 33 5 65 35 6 60 43 7 67 46 8 55 9 56 10 11 90 24 Do reflexes slow down with age? Are there gender differences? A portion of the data is shown here.

Scatterplots with Age, Gender
Does age seem related? How about Gender?

Correlation, Regression
SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 20 Correlations  Score Age 1 Gender 0.1406 Coefficients Standard Error t Stat P-value Intercept E-14 Age E-07 Gender Age is related, gender is not.

Interaction Term Y X1 X2 X1*X2 Obs Score Age Gender Age* 1 80 25 2 82 28 3 75 32 4 70 33 5 65 35 6 60 43 7 67 46 .. 11 90 24 12 87 A 2-way interaction term is the product of the two variables.

Regression with Interaction
SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 20 How do we interpret the coefficient for the interaction term? Coefficients Standard Error t Stat P-value Intercept 1.49E-11 Age Gender Age*Gender

Meaning of Interaction
X1 and X2 are said to interact with each other if the impact of X1 on y changes as the value of X2 changes. In this example, the impact of age (X1) on reflexes (y) is different for males and females (changing values of X2). Hence age and gender are said to interact. Explain how this is different from multicollinearity.

IV. Heteroscedasticity
Consider the water levels in Lake Lanier. There is a trend that can be used to forecast. However, the variability around the trendline is not consistent. The increase in variation makes the prediction margin of error unreliable.

Example 2: Income and Spending
As income grows, the ability to spend on luxury goods grows with it, and so does the variation in how much is actually spent. Once again, forecasts become less reliable due to changing variation (heteroscedasticity).

Solution When heteroscedasticity is identified, data may need to be transformed (change to a log scale, for instance) to reduce its impact. The type of transformation needed depends on the data.

Similar presentations