Download presentation

Presentation is loading. Please wait.

Published byKarley Tomson Modified over 2 years ago

1
1 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS X Y XiXi 11 1 + 2 X i Y = 1 + 2 X We will now apply the maximum likelihood principle to regression analysis, using the simple linear model Y = 1 + 2 X + u.

2
2 The black marker shows the value that Y would have if X were equal to X i and if there were no disturbance term. X Y XiXi 11 1 + 2 X i Y = 1 + 2 X MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

3
3 However we will assume that there is a disturbance term in the model and that it has a normal distribution as shown. X Y XiXi 11 1 + 2 X i Y = 1 + 2 X MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

4
4 Relative to the black marker, the curve represents the ex ante distribution for u, that is, its potential distribution before the observation is generated. Ex post, of course, it is fixed at some specific value. X Y XiXi 11 1 + 2 X i Y = 1 + 2 X MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

5
5 Relative to the horizontal axis, the curve also represents the ex ante distribution for Y for that observation, that is, conditional on X = X i. X Y XiXi 11 1 + 2 X i Y = 1 + 2 X MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

6
6 Potential values of Y close to 1 + 2 X i will have relatively large densities... X Y XiXi 11 1 + 2 X i Y = 1 + 2 X MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

7
X Y XiXi 11 1 + 2 X i Y = 1 + 2 X 7... while potential values of Y relatively far from 1 + 2 X i will have small ones. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

8
8 The mean value of the distribution of Y i is 1 + 2 X i. Its standard deviation is , the standard deviation of the disturbance term. X Y XiXi 11 1 + 2 X i Y = 1 + 2 X MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

9
X Y XiXi 11 1 + 2 X i Y = 1 + 2 X 9 Hence the density function for the ex ante distribution of Y i is as shown. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

10
10 The joint density function for the observations on Y is the product of their individual densities. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

11
11 Now, taking 1, 2 and as our choice variables, and taking the data on Y and X as given, we can re-interpret this function as the likelihood function for 1, 2, and . MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

12
12 We will choose 1, 2, and so as to maximize the likelihood, given the data on Y and X. As usual, it is easier to do this indirectly, maximizing the log-likelihood instead. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

13
13 As usual, the first step is to decompose the expression as the sum of the logarithms of the factors. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

14
14 Then we split the logarithm of each factor into two components. The first component is the same in each case. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

15
15 Hence the log-likelihood simplifies as shown. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS where

16
16 To maximize the log-likelihood, we need to minimize Z. But choosing estimators of 1 and 2 to minimize Z is exactly what we did when we derived the least squares regression coefficients. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS where

17
17 Thus, for this regression model, the maximum likelihood estimators of 1 and 2 are identical to the least squares estimators. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS where

18
18 As a consequence, Z will be the sum of the squares of the least squares residuals. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS where

19
19 To obtain the maximum likelihood estimator of , it is convenient to rearrange the log- likelihood function as shown. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

20
20 Differentiating it with respect to , we obtain the expression shown. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

21
21 The first order condition for a maximum requires this to be equal to zero. Hence the maximum likelihood estimator of the variance is the sum of the squares of the residuals divided by n. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

22
22 Note that this is biased for finite samples. To obtain an unbiased estimator, we should divide by n–k, where k is the number of parameters, in this case 2. However, the bias disappears as the sample size becomes large. MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS

23
Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section 10.6 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course 20 Elements of Econometrics

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google