1. (1)Combine the correlated variables. 1 In this sequence, we look at four possible indirect methods for alleviating a problem of multicollinearity. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

2. 2 First, if the correlated variables are similar conceptually, it may be reasonable to combine them into some overall index. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (1)Combine the correlated variables.

3. 3 That is precisely what has been done with the three cognitive ASVAB variables. ASVABC has been calculated as a weighted average of scores on subtests: ASVAB02 (arithmetic reasoning), ASVAB03 (word knowledge), and ASVAB04 (paragraph comprehension).. reg EARNINGS S EXP EXPSQ MALE ASVABC Source | SS df MS Number of obs = 540 -------------+------------------------------ F( 5, 534) = 37.24 Model | 28957.3532 5 5791.47063 Prob > F = 0.0000 Residual | 83052.8779 534 155.529734 R-squared = 0.2585 -------------+------------------------------ Adj R-squared = 0.2516 Total | 112010.231 539 207.811189 Root MSE = 12.471 ------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- S | 2.031419.296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828.6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ |.0130223.021334 0.61 0.542 -.0288866.0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC |.2447687.0714294 3.43 0.001.1044516.3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535 ------------------------------------------------------------------------------ POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

4. 4 The three components are highly correlated and by combining them as a weighted average, rather than using them individually, one avoids a potential problem of multicollinearity. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. reg EARNINGS S EXP EXPSQ MALE ASVABC Source | SS df MS Number of obs = 540 -------------+------------------------------ F( 5, 534) = 37.24 Model | 28957.3532 5 5791.47063 Prob > F = 0.0000 Residual | 83052.8779 534 155.529734 R-squared = 0.2585 -------------+------------------------------ Adj R-squared = 0.2516 Total | 112010.231 539 207.811189 Root MSE = 12.471 ------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- S | 2.031419.296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828.6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ |.0130223.021334 0.61 0.542 -.0288866.0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC |.2447687.0714294 3.43 0.001.1044516.3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535 ------------------------------------------------------------------------------

5. (2)Drop some of the correlated variables. 5 Dropping some of the correlated variables, if they have insignificant coefficients, may alleviate multicollinearity. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

6. 6 However, this approach to multicollinearity is dangerous. It is possible that some of the variables with insignificant coefficients really do belong in the model and that the only reason their coefficients are insignificant is because there is a problem of multicollinearity. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (2)Drop some of the correlated variables.

7. 7 If that is the case, their omission may cause omitted variable bias, to be discussed in Chapter 6. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (2)Drop some of the correlated variables.

8. 8 A further way of dealing with the problem of multicollinearity is to use extraneous information, if available, concerning the coefficient of one of the variables. (3)Empirical restriction POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

9. 9 For example, suppose that Y in the equation above is the demand for a category of consumer expenditure, X is aggregate disposable personal income, and P is a price index for the category. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (3)Empirical restriction

10. 10 To fit a model of this type you would use time series data. If X and P are highly correlated, which is often the case with time series variables, the problem of multicollinearity might be eliminated in the following way. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (3)Empirical restriction

11. 11 Obtain data on income and expenditure on the category from a household survey and regress Y' on X'. (The ' marks are to indicate that the data are household data, not aggregate data.) POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (3)Empirical restriction

12. 12 This is a simple regression because there will be relatively little variation in the price paid by the households. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (3)Empirical restriction

13. 13 Now substitute b' for  2 in the time series model. Subtract b' X from both sides, and regress Z = Y – b' X on price. This is a simple regression, so multicollinearity has been eliminated. 2 2 2 POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (3)Empirical restriction

14. 14 There are some problems with this technique. First, the  2 coefficients may be conceptually different in time series and cross-section contexts. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (3)Empirical restriction

15. 15 Second, since we subtract the estimated income component b' X, not the true income component  2 X, from Y when constructing Z, we have introduced an element of measurement error in the dependent variable. 2 POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (3)Empirical restriction

16. 16 The last, but by no means least, indirect method for alleviating multicollinearity is the use of a theoretical restriction, which is defined as a hypothetical relationship among the parameters of a regression model. (4)Theoretical restriction POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

17. 17 It will be explained using an educational attainment model as an example. Suppose that we hypothesize that highest grade completed, S, depends on ASVABC, and highest grade completed by the respondent's mother and father, SM and SF, respectively. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (4)Theoretical restriction

18. 18 A one-point increase in ASVABC increases S by 0.13 years.. reg S ASVABC SM SF Source | SS df MS Number of obs = 540 -------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686 -------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943 ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1257087.0098533 12.76 0.000.1063528.1450646 SM |.0492424.0390901 1.26 0.208 -.027546.1260309 SF |.1076825.0309522 3.48 0.001.04688.1684851 _cons | 5.370631.4882155 11.00 0.000 4.41158 6.329681 ------------------------------------------------------------------------------ POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

19. 19 S increases by 0.05 years for every extra year of schooling of the mother and 0.11 years for every extra year of schooling of the father.. reg S ASVABC SM SF Source | SS df MS Number of obs = 540 -------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686 -------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943 ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1257087.0098533 12.76 0.000.1063528.1450646 SM |.0492424.0390901 1.26 0.208 -.027546.1260309 SF |.1076825.0309522 3.48 0.001.04688.1684851 _cons | 5.370631.4882155 11.00 0.000 4.41158 6.329681 ------------------------------------------------------------------------------ POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

20. 20 Mother's education is generally held to be at least, if not more, important than father's education for educational attainment, so this outcome is unexpected. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. reg S ASVABC SM SF Source | SS df MS Number of obs = 540 -------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686 -------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943 ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1257087.0098533 12.76 0.000.1063528.1450646 SM |.0492424.0390901 1.26 0.208 -.027546.1260309 SF |.1076825.0309522 3.48 0.001.04688.1684851 _cons | 5.370631.4882155 11.00 0.000 4.41158 6.329681 ------------------------------------------------------------------------------

21. 21 It is also surprising that the coefficient of SM is not significant, even at the 5 percent level, using a one-sided test. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. reg S ASVABC SM SF Source | SS df MS Number of obs = 540 -------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686 -------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943 ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1257087.0098533 12.76 0.000.1063528.1450646 SM |.0492424.0390901 1.26 0.208 -.027546.1260309 SF |.1076825.0309522 3.48 0.001.04688.1684851 _cons | 5.370631.4882155 11.00 0.000 4.41158 6.329681 ------------------------------------------------------------------------------

22. . reg S ASVABC SM SF Source | SS df MS Number of obs = 540 -------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686 -------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943 ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1257087.0098533 12.76 0.000.1063528.1450646 SM |.0492424.0390901 1.26 0.208 -.027546.1260309 SF |.1076825.0309522 3.48 0.001.04688.1684851 _cons | 5.370631.4882155 11.00 0.000 4.41158 6.329681 ------------------------------------------------------------------------------ 22 However assortive mating leads to correlation between SM and SF and the regression appears to be suffering from multicollinearity.. cor SM SF (obs=540) | SM SF --------+------------------ SM | 1.0000 SF | 0.6241 1.0000 POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

23. 23 Suppose that we hypothesize that mother's and father's education are equally important. We can then impose the restriction  3 =  4. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (4)Theoretical restriction

24. 24 This allows us to rewrite the equation as shown. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (4)Theoretical restriction

25. 25 Defining SP to be the sum of SM and SF, the equation may be rewritten as shown. The problem caused by the correlation between SM and SF has been eliminated. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY (4)Theoretical restriction

26. . g SP=SM+SF. reg S ASVABC SP Source | SS df MS Number of obs = 540 -------------+------------------------------ F( 2, 537) = 156.04 Model | 1177.98338 2 588.991689 Prob > F = 0.0000 Residual | 2026.99996 537 3.77467403 R-squared = 0.3675 -------------+------------------------------ Adj R-squared = 0.3652 Total | 3204.98333 539 5.94616574 Root MSE = 1.9429 ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1253106.0098434 12.73 0.000.1059743.1446469 SP |.0828368.0164247 5.04 0.000.0505722.1151014 _cons | 5.29617.4817972 10.99 0.000 4.349731 6.242608 ------------------------------------------------------------------------------ 26 The estimate of  3 is now 0.083. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

27. . g SP=SM+SF. reg S ASVABC SP ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1253106.0098434 12.73 0.000.1059743.1446469 SP |.0828368.0164247 5.04 0.000.0505722.1151014 _cons | 5.29617.4817972 10.99 0.000 4.349731 6.242608 ------------------------------------------------------------------------------ 27 Not surprisingly, this is a compromise between the coefficients of SM and SF in the previous specification. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. reg S ASVABC SM SF ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1257087.0098533 12.76 0.000.1063528.1450646 SM |.0492424.0390901 1.26 0.208 -.027546.1260309 SF |.1076825.0309522 3.48 0.001.04688.1684851 _cons | 5.370631.4882155 11.00 0.000 4.41158 6.329681 ------------------------------------------------------------------------------

28. 28 The standard error of SP is much smaller than those of SM and SF. The use of the restriction has led to a large gain in efficiency and the problem of multicollinearity has been eliminated. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY. g SP=SM+SF. reg S ASVABC SP ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1253106.0098434 12.73 0.000.1059743.1446469 SP |.0828368.0164247 5.04 0.000.0505722.1151014 _cons | 5.29617.4817972 10.99 0.000 4.349731 6.242608 ------------------------------------------------------------------------------. reg S ASVABC SM SF ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1257087.0098533 12.76 0.000.1063528.1450646 SM |.0492424.0390901 1.26 0.208 -.027546.1260309 SF |.1076825.0309522 3.48 0.001.04688.1684851 _cons | 5.370631.4882155 11.00 0.000 4.41158 6.329681 ------------------------------------------------------------------------------

29. . g SP=SM+SF. reg S ASVABC SP ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1253106.0098434 12.73 0.000.1059743.1446469 SP |.0828368.0164247 5.04 0.000.0505722.1151014 _cons | 5.29617.4817972 10.99 0.000 4.349731 6.242608 ------------------------------------------------------------------------------. reg S ASVABC SM SF ------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- ASVABC |.1257087.0098533 12.76 0.000.1063528.1450646 SM |.0492424.0390901 1.26 0.208 -.027546.1260309 SF |.1076825.0309522 3.48 0.001.04688.1684851 _cons | 5.370631.4882155 11.00 0.000 4.41158 6.329681 ------------------------------------------------------------------------------ 29 The t statistic is very high. Thus it would appear that imposing the restriction has improved the regression results. However, the restriction may not be valid. We should test it. Testing theoretical restrictions is one of the topics in Chapter 6. POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY

30. Copyright Christopher Dougherty 2012. These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section 3.4 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre http://www.oup.com/uk/orc/bin/9780199567089/http://www.oup.com/uk/orc/bin/9780199567089/. Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx or the University of London International Programmes distance learning course EC2020 Elements of Econometrics www.londoninternational.ac.uk/lsewww.londoninternational.ac.uk/lse. 2012.10.28