Christopher Dougherty EC220 - Introduction to econometrics (chapter 1) Slideshow: simple regression model Original citation: Dougherty, C. (2012) EC220 - Introduction to econometrics (chapter 1). [Teaching Resource] © 2012 The Author This version available at: Available in LSE Learning Resources Online: May 2012 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms

1 Y SIMPLE REGRESSION MODEL Suppose that a variable Y is a linear function of another variable X, with unknown parameters  1 and  2 that we wish to estimate. 11 X X1X1 X2X2 X3X3 X4X4

Suppose that we have a sample of 4 observations with X values as shown. SIMPLE REGRESSION MODEL 2 11 Y X X1X1 X2X2 X3X3 X4X4

If the relationship were an exact one, the observations would lie on a straight line and we would have no trouble obtaining accurate estimates of  1 and  2. Q1Q1 Q2Q2 Q3Q3 Q4Q4 SIMPLE REGRESSION MODEL 3 11 Y X X1X1 X2X2 X3X3 X4X4

P4P4 In practice, most economic relationships are not exact and the actual values of Y are different from those corresponding to the straight line. P3P3 P2P2 P1P1 Q1Q1 Q2Q2 Q3Q3 Q4Q4 SIMPLE REGRESSION MODEL 4 11 Y X X1X1 X2X2 X3X3 X4X4

P4P4 To allow for such divergences, we will write the model as Y =  1 +  2 X + u, where u is a disturbance term. P3P3 P2P2 P1P1 Q1Q1 Q2Q2 Q3Q3 Q4Q4 SIMPLE REGRESSION MODEL 5 11 Y X X1X1 X2X2 X3X3 X4X4

P4P4 Each value of Y thus has a nonrandom component,  1 +  2 X, and a random component, u. The first observation has been decomposed into these two components. P3P3 P2P2 P1P1 Q1Q1 Q2Q2 Q3Q3 Q4Q4 u1u1 SIMPLE REGRESSION MODEL 6 11 Y X X1X1 X2X2 X3X3 X4X4

P4P4 In practice we can see only the P points. P3P3 P2P2 P1P1 SIMPLE REGRESSION MODEL 7 Y X X1X1 X2X2 X3X3 X4X4

P4P4 Obviously, we can use the P points to draw a line which is an approximation to the line Y =  1 +  2 X. If we write this line Y = b 1 + b 2 X, b 1 is an estimate of  1 and b 2 is an estimate of  2. P3P3 P2P2 P1P1 ^ SIMPLE REGRESSION MODEL 8 b1b1 Y X X1X1 X2X2 X3X3 X4X4

P4P4 The line is called the fitted model and the values of Y predicted by it are called the fitted values of Y. They are given by the heights of the R points. P3P3 P2P2 P1P1 R1R1 R2R2 R3R3 R4R4 SIMPLE REGRESSION MODEL 9 b1b1 (fitted value) Y (actual value) Y X X1X1 X2X2 X3X3 X4X4

P4P4 X X1X1 X2X2 X3X3 X4X4 The discrepancies between the actual and fitted values of Y are known as the residuals. P3P3 P2P2 P1P1 R1R1 R2R2 R3R3 R4R4 (residual) e1e1 e2e2 e3e3 e4e4 SIMPLE REGRESSION MODEL 10 b1b1 (fitted value) Y (actual value) Y

P4P4 Note that the values of the residuals are not the same as the values of the disturbance term. The diagram now shows the true unknown relationship as well as the fitted line. P3P3 P2P2 P1P1 R1R1 R2R2 R3R3 R4R4 b1b1 SIMPLE REGRESSION MODEL 11 11 (fitted value) Y (actual value) Y X X1X1 X2X2 X3X3 X4X4

P4P4 The disturbance term in each observation is responsible for the divergence between the nonrandom component of the true relationship and the actual observation. P3P3 P2P2 P1P1 SIMPLE REGRESSION MODEL 12 Q2Q2 Q1Q1 Q3Q3 Q4Q4 11 b1b1 (fitted value) Y (actual value) Y X X1X1 X2X2 X3X3 X4X4

P4P4 The residuals are the discrepancies between the actual and the fitted values. P3P3 P2P2 P1P1 R1R1 R2R2 R3R3 R4R4 SIMPLE REGRESSION MODEL 13 11 b1b1 (fitted value) Y (actual value) Y X X1X1 X2X2 X3X3 X4X4

P4P4 If the fit is a good one, the residuals and the values of the disturbance term will be similar, but they must be kept apart conceptually. P3P3 P2P2 P1P1 R1R1 R2R2 R3R3 R4R4 SIMPLE REGRESSION MODEL 14 11 b1b1 (fitted value) Y (actual value) Y X X1X1 X2X2 X3X3 X4X4

P4P4 Both of these lines will be used in our analysis. Each permits a decomposition of the value of Y. The decompositions will be illustrated with the fourth observation. SIMPLE REGRESSION MODEL 15 Q4Q4 u4u4 11 b1b1 (fitted value) Y (actual value) Y X X1X1 X2X2 X3X3 X4X4

P4P4 Using the theoretical relationship, Y can be decomposed into its nonstochastic component  1 +  2 X and its random component u. SIMPLE REGRESSION MODEL 16 Q4Q4 u4u4 11 b1b1 (fitted value) Y (actual value) Y X X1X1 X2X2 X3X3 X4X4

P4P4 This is a theoretical decomposition because we do not know the values of  1 or  2, or the values of the disturbance term. We shall use it in our analysis of the properties of the regression coefficients. SIMPLE REGRESSION MODEL 17 Q4Q4 u4u4 11 b1b1 (fitted value) Y (actual value) Y X X1X1 X2X2 X3X3 X4X4

P4P4 The other decomposition is with reference to the fitted line. In each observation, the actual value of Y is equal to the fitted value plus the residual. This is an operational decomposition which we will use for practical purposes. SIMPLE REGRESSION MODEL 18 e4e4 R4R4 11 b1b1 Y (actual value) (fitted value) Y X X1X1 X2X2 X3X3 X4X4

SIMPLE REGRESSION MODEL To begin with, we will draw the fitted line so as to minimize the sum of the squares of the residuals, RSS. This is described as the least squares criterion. 19 Least squares criterion: Minimize RSS (residual sum of squares), where

SIMPLE REGRESSION MODEL Why the squares of the residuals? Why not just minimize the sum of the residuals? Least squares criterion: Why not minimize 20 Minimize RSS (residual sum of squares), where

P4P4 The answer is that you would get an apparently perfect fit by drawing a horizontal line through the mean value of Y. The sum of the residuals would be zero. P3P3 P2P2 P1P1 SIMPLE REGRESSION MODEL Y 21 X X1X1 X2X2 X3X3 X4X4 Y

P4P4 You must prevent negative residuals from cancelling positive ones, and one way to do this is to use the squares of the residuals. P3P3 P2P2 P1P1 SIMPLE REGRESSION MODEL 22 X X1X1 X2X2 X3X3 X4X4 Y Y

P4P4 Of course there are other ways of dealing with the problem. The least squares criterion has the attraction that the estimators derived with it have desirable properties, provided that certain conditions are satisfied. P3P3 P2P2 P1P1 SIMPLE REGRESSION MODEL 23 X X1X1 X2X2 X3X3 X4X4 Y Y

P4P4 The next sequence shows how the least squares criterion is used to calculate the coefficients of the fitted line. P3P3 P2P2 P1P1 SIMPLE REGRESSION MODEL 24 X X1X1 X2X2 X3X3 X4X4 Y Y

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section 1.2 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own and who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course 20 Elements of Econometrics