POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY 1 What can you do about multicollinearity if you encounter it? We will discuss some possible measures, looking at the model with two explanatory variables.

2 Before doing this, two important points should be emphasized. First, multicollinearity does not cause the regression coefficients to be biased. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

3 The problem is that they have unsatisfactorily large variances. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

4 Second, the standard errors and t tests remain valid. The standard errors are larger than they would have been in the absence of multicollinearity, warning us that the regression estimates are erratic. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

5 The problem of multicollinearity is caused by the variances of the coefficients being unsatisfactorily large. In this sequence, we will look at possible direct methods of reducing them. In the next sequence, we will look at possible indirect methods. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

(1)Reduce by including further relevant variables in the model. 6 We will focus on the slope coefficient and look at the various components of its variance. We might be able to reduce the variance by bringing more variables into the model and reducing  u 2, the variance of the disturbance term. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

7 The estimator of the variance of the disturbance term is the residual sum of squares divided by n – k, where n is the number of observations (540) and k is the number of parameters (4). Here it is reg EARNINGS S EXP EXPSQ Source | SS df MS Number of obs = F( 3, 536) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | _cons | POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

8. reg EARNINGS S EXP EXPSQ MALE ASVABC Source | SS df MS Number of obs = F( 5, 534) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | We now add two new variables that are often found to be determinants of earnings: MALE, sex of respondent, and ASVABC, the composite score on the cognitive tests in the Armed Services Vocational Aptitude Battery. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

9 MALE is a qualitative variable and the treatment of such variables will be explained in Chapter 5. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. reg EARNINGS S EXP EXPSQ MALE ASVABC Source | SS df MS Number of obs = F( 5, 534) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons |

. reg EARNINGS S EXP EXPSQ MALE ASVABC Source | SS df MS Number of obs = F( 5, 534) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | Both MALE and ASVABC have coefficients significant at the 0.1% level. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ Source | SS df MS Number of obs = F( 3, 536) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = However they account for only a small proportion of the variance in earnings and the reduction in the estimate of the variance of the disturbance term is likewise small. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. reg EARNINGS S EXP EXPSQ MALE ASVABC Source | SS df MS Number of obs = F( 5, 534) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE =

12 As a consequence the impact on the standard errors of EXP and EXPSQ is negligible. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. reg EARNINGS S EXP EXPSQ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons |

. reg EARNINGS S EXP EXPSQ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | Note how unstable the coefficients are. This is often a sign of multicollinearity. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

14 Note also that the standard error of the coefficient of S has actually increased. This is attributable to the correlation of 0.58 between S and ASVABC. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. reg EARNINGS S EXP EXPSQ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | cor S ASVABC (obs=540) | S ASVABC S | ASVABC |

15 This is a common problem with this approach to attempting to reduce the problem of multicollinearity. If the new variables are linearly related to one or more of the variables already in the equation, their inclusion may make the problem of multicollinearity worse.. reg EARNINGS S EXP EXPSQ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | cor S ASVABC (obs=540) | S ASVABC S | ASVABC | POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

16 The next factor to look at is n, the number of observations. If you are working with cross- section data (individuals, households, enterprises, etc) and you are undertaking a survey, you could increase the size of the sample by negotiating a bigger budget. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (2)Increase the number of observations. Surveys: increase the budget, use clustering.

17 Alternatively, you could make a fixed budget go further by using a technique known as clustering. You divide the country geographically by zip code or postal area. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (2)Increase the number of observations. Surveys: increase the budget, use clustering.

18 You select a number of these randomly, perhaps using stratified random sampling to make sure that metropolitan, other urban, and rural areas are properly represented. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (2)Increase the number of observations. Surveys: increase the budget, use clustering.

19 You then confine the survey to the areas selected. This reduces the travel time and cost of the fieldworkers, allowing them to interview a greater number of respondents. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (2)Increase the number of observations. Surveys: increase the budget, use clustering.

(2)Increase the number of observations. Surveys: increase the budget, use clustering. Time series: use quarterly instead of annual data. 20 If you are working with time series data, you may be able to increase the sample by working with shorter time intervals for the data, for example quarterly or even monthly data instead of annual data. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC Source | SS df MS Number of obs = F( 5, 2708) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | Here is the result of running the regression with all 2,714 observations in the EAEF data set. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | Comparing this result with that using Data Set 21, we see that the standard errors are much smaller, as expected. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

23 As a consequence, the t statistics of the variables are higher. However, the correlation between EXP and EXPSQ is as high as in the smaller sample and the increase in the sample size has not been large enough to have much impact on the problem of multicollinearity.. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

24 The coefficients of EXP and EXPSQ both still have unexpected signs since we expect the coefficient of EXP to be positive and that of EXPSQ to be negative, reflecting diminishing returns.. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

25 The EXPSQ coefficient has a rather large t statistic, which is a matter of concern. We could assume that this has occurred as a matter of chance. Alternatively, it might be an indication that the model is misspecified.. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

26 As we will see in the next and subsequent chapters, there are good reasons for supposing that the dependent variable in an earnings function should be the logarithm of earnings, rather than earnings in linear form.. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | EXPSQ | MALE | ASVABC | _cons | POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

27 A third possible way of reducing the problem of multicollinearity might be to increase the variation in the explanatory variables. This is possible only at the design stage of a survey. (3)Increase MSD(X 2 ). POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

28 For example, if you were planning a household survey with the aim of investigating how expenditure patterns vary with income, you should make sure that the sample included relatively rich and relatively poor households as well as middle-income households. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (3)Increase MSD(X 2 ).

29 Another possibility might be to reduce the correlation between the explanatory variables. Again, this is possible only at the design stage of a survey and even then it is seldom possible with economic data. POSSIBLE DIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (4)Reduce.

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section 3.4 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course EC2020 Elements of Econometrics