Christopher Dougherty EC220 - Introduction to econometrics (chapter 10) Slideshow: maximum likelihood estimation of regression coefficients Original citation:

Slides:



Advertisements
Similar presentations
Christopher Dougherty EC220 - Introduction to econometrics (chapter 9) Slideshow: two-stage least squares Original citation: Dougherty, C. (2012) EC220.
Advertisements

Christopher Dougherty EC220 - Introduction to econometrics (chapter 11) Slideshow: model c assumptions Original citation: Dougherty, C. (2012) EC220 -
EC220 - Introduction to econometrics (chapter 8)
Christopher Dougherty EC220 - Introduction to econometrics (chapter 8) Slideshow: model b: properties of the regression coefficients Original citation:
EC220 - Introduction to econometrics (chapter 1)
1 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS X Y XiXi 11  1  +  2 X i Y =  1  +  2 X We will now apply the maximum likelihood principle.
EC220 - Introduction to econometrics (chapter 3)
EC220 - Introduction to econometrics (chapter 4)
EC220 - Introduction to econometrics (review chapter)
EC220 - Introduction to econometrics (chapter 10)
Christopher Dougherty EC220 - Introduction to econometrics (chapter 2) Slideshow: a Monte Carlo experiment Original citation: Dougherty, C. (2012) EC220.
Christopher Dougherty EC220 - Introduction to econometrics (chapter 10) Slideshow: introduction to maximum likelihood estimation Original citation: Dougherty,
Christopher Dougherty EC220 - Introduction to econometrics (chapter 11) Slideshow: adaptive expectations Original citation: Dougherty, C. (2012) EC220.
1 THE DISTURBANCE TERM IN LOGARITHMIC MODELS Thus far, nothing has been said about the disturbance term in nonlinear regression models.
EC220 - Introduction to econometrics (chapter 7)
1 XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density Sample of n observations X 1, X 2,..., X n : potential.
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: asymptotic properties of estimators: plims and consistency Original.
Christopher Dougherty EC220 - Introduction to econometrics (chapter 12) Slideshow: dynamic model specification Original citation: Dougherty, C. (2012)
Christopher Dougherty EC220 - Introduction to econometrics (chapter 2) Slideshow: testing a hypothesis relating to a regression coefficient Original citation:
1 THE NORMAL DISTRIBUTION In the analysis so far, we have discussed the mean and the variance of a distribution of a random variable, but we have not said.
EC220 - Introduction to econometrics (chapter 7)
1 PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE red This sequence provides an example of a discrete random variable. Suppose that you.
Random effects estimation RANDOM EFFECTS REGRESSIONS When the observed variables of interest are constant for each individual, a fixed effects regression.
MEASUREMENT ERROR 1 In this sequence we will investigate the consequences of measurement errors in the variables in a regression model. To keep the analysis.
1 ASSUMPTIONS FOR MODEL C: REGRESSIONS WITH TIME SERIES DATA Assumptions C.1, C.3, C.4, C.5, and C.8, and the consequences of their violations are the.
EC220 - Introduction to econometrics (chapter 2)
EC220 - Introduction to econometrics (chapter 9)
EXPECTED VALUE OF A RANDOM VARIABLE 1 The expected value of a random variable, also known as its population mean, is the weighted average of its possible.
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: expected value of a function of a random variable Original citation:
Christopher Dougherty EC220 - Introduction to econometrics (chapter 6) Slideshow: variable misspecification iii: consequences for diagnostics Original.
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: confidence intervals Original citation: Dougherty, C. (2012) EC220.
EC220 - Introduction to econometrics (review chapter)
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: continuous random variables Original citation: Dougherty, C. (2012)
1 A MONTE CARLO EXPERIMENT In the previous slideshow, we saw that the error term is responsible for the variations of b 2 around its fixed component 
Christopher Dougherty EC220 - Introduction to econometrics (chapter 3) Slideshow: prediction Original citation: Dougherty, C. (2012) EC220 - Introduction.
Christopher Dougherty EC220 - Introduction to econometrics (chapter 3) Slideshow: precision of the multiple regression coefficients Original citation:
Christopher Dougherty EC220 - Introduction to econometrics (chapter 4) Slideshow: semilogarithmic models Original citation: Dougherty, C. (2012) EC220.
Christopher Dougherty EC220 - Introduction to econometrics (chapter 4) Slideshow: nonlinear regression Original citation: Dougherty, C. (2012) EC220 -
DERIVING LINEAR REGRESSION COEFFICIENTS
EC220 - Introduction to econometrics (chapter 12)
Christopher Dougherty EC220 - Introduction to econometrics (chapter 5) Slideshow: Chow test Original citation: Dougherty, C. (2012) EC220 - Introduction.
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: the normal distribution Original citation: Dougherty, C. (2012)
1 In a second variation, we shall consider the model shown above. x is the rate of growth of productivity, assumed to be exogenous. w is now hypothesized.
EC220 - Introduction to econometrics (review chapter)
1 UNBIASEDNESS AND EFFICIENCY Much of the analysis in this course will be concerned with three properties of estimators: unbiasedness, efficiency, and.
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: sampling and estimators Original citation: Dougherty, C. (2012)
Christopher Dougherty EC220 - Introduction to econometrics (chapter 12) Slideshow: autocorrelation, partial adjustment, and adaptive expectations Original.
THE DUMMY VARIABLE TRAP 1 Suppose that you have a regression model with Y depending on a set of ordinary variables X 2,..., X k and a qualitative variable.
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: conflicts between unbiasedness and minimum variance Original citation:
Christopher Dougherty EC220 - Introduction to econometrics (chapter 8) Slideshow: measurement error Original citation: Dougherty, C. (2012) EC220 - Introduction.
THE FIXED AND RANDOM COMPONENTS OF A RANDOM VARIABLE 1 In this short sequence we shall decompose a random variable X into its fixed and random components.
Christopher Dougherty EC220 - Introduction to econometrics (chapter 11) Slideshow: Friedman Original citation: Dougherty, C. (2012) EC220 - Introduction.
ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE 1 This sequence derives an alternative expression for the population variance of a random variable. It provides.
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
EC220 - Introduction to econometrics (chapter 8)
Christopher Dougherty EC220 - Introduction to econometrics (chapter 12) Slideshow: footnote: the Cochrane-Orcutt iterative process Original citation: Dougherty,
A.1The model is linear in parameters and correctly specified. PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS 1 Moving from the simple to the multiple.
Christopher Dougherty EC220 - Introduction to econometrics (chapter 9) Slideshow: instrumental variable estimation: variation Original citation: Dougherty,
Christopher Dougherty EC220 - Introduction to econometrics (chapter 6) Slideshow: multiple restrictions and zero restrictions Original citation: Dougherty,
1 We will now look at the properties of the OLS regression estimators with the assumptions of Model B. We will do this within the context of the simple.
1 Y SIMPLE REGRESSION MODEL Suppose that a variable Y is a linear function of another variable X, with unknown parameters  1 and  2 that we wish to estimate.
1 We will continue with a variation on the basic model. We will now hypothesize that p is a function of m, the rate of growth of the money supply, as well.
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: alternative expression for population variance Original citation:
INSTRUMENTAL VARIABLES 1 Suppose that you have a model in which Y is determined by X but you have reason to believe that Assumption B.7 is invalid and.
1 ESTIMATORS OF VARIANCE, COVARIANCE, AND CORRELATION We have seen that the variance of a random variable X is given by the expression above. Variance.
Christopher Dougherty EC220 - Introduction to econometrics (chapter 2) Slideshow: confidence intervals Original citation: Dougherty, C. (2012) EC220 -
Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: independence of two random variables Original citation: Dougherty,
Christopher Dougherty EC220 - Introduction to econometrics (chapter 1) Slideshow: simple regression model Original citation: Dougherty, C. (2012) EC220.
FOOTNOTE: THE COCHRANE–ORCUTT ITERATIVE PROCESS 1 We saw in the previous sequence that AR(1) autocorrelation could be eliminated by a simple manipulation.
Introduction to Econometrics, 5th edition
Presentation transcript:

Christopher Dougherty EC220 - Introduction to econometrics (chapter 10) Slideshow: maximum likelihood estimation of regression coefficients Original citation: Dougherty, C. (2012) EC220 - Introduction to econometrics (chapter 10). [Teaching Resource] © 2012 The Author This version available at: Available in LSE Learning Resources Online: May 2012 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms

1 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS X Y XiXi 11  1  +  2 X i Y =  1  +  2 X We will now apply the maximum likelihood principle to regression analysis, using the simple linear model Y =  1 +  2 X + u.

2 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS The black marker shows the value that Y would have if X were equal to X i and if there were no disturbance term. X Y XiXi 11  1  +  2 X i Y =  1  +  2 X

3 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS However we will assume that there is a disturbance term in the model and that it has a normal distribution as shown. X Y XiXi 11  1  +  2 X i Y =  1  +  2 X

4 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Relative to the black marker, the curve represents the ex ante distribution for u, that is, its potential distribution before the observation is generated. Ex post, of course, it is fixed at some specific value. X Y XiXi 11  1  +  2 X i Y =  1  +  2 X

5 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Relative to the horizontal axis, the curve also represents the ex ante distribution for Y for that observation, that is, conditional on X = X i. X Y XiXi 11  1  +  2 X i Y =  1  +  2 X

6 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Potential values of Y close to  1 +  2 X i will have relatively large densities... X Y XiXi 11  1  +  2 X i Y =  1  +  2 X

X Y XiXi 11  1  +  2 X i Y =  1  +  2 X 7 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS... while potential values of Y relatively far from  1 +  2 X i will have small ones.

8 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS The mean value of the distribution of Y i is  1 +  2 X i. Its standard deviation is , the standard deviation of the disturbance term. X Y XiXi 11  1  +  2 X i Y =  1  +  2 X

9 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Hence the density function for the ex ante distribution of Y i is as shown. X Y XiXi 11  1  +  2 X i Y =  1  +  2 X

10 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS The joint density function for the observations on Y is the product of their individual densities.

11 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Now, taking  1,  2 and  as our choice variables, and taking the data on Y and X as given, we can re-interpret this function as the likelihood function for  1,  2, and .

12 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS We will choose  1,  2, and  so as to maximize the likelihood, given the data on Y and X. As usual, it is easier to do this indirectly, maximizing the log-likelihood instead.

13 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS As usual, the first step is to decompose the expression as the sum of the logarithms of the factors.

14 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Then we split the logarithm of each factor into two components. The first component is the same in each case.

15 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Hence the log-likelihood simplifies as shown.

16 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS To maximize the log-likelihood, we need to minimize Z. But choosing estimators of  1 and  2 to minimize Z is exactly what we did when we derived the least squares regression coefficients.

17 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Thus, for this regression model, the maximum likelihood estimators of  1 and  2 are identical to the least squares estimators.

18 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS As a consequence, Z will be the sum of the squares of the least squares residuals.

19 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS To obtain the maximum likelihood estimator of , it is convenient to rearrange the log- likelihood function as shown.

20 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Differentiating it with respect to , we obtain the expression shown.

21 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS The first order condition for a maximum requires this to be equal to zero. Hence the maximum likelihood estimator of the variance is the sum of the squares of the residuals divided by n.

22 MAXIMUM LIKELIHOOD ESTIMATION OF REGRESSION COEFFICIENTS Note that this is biased for finite samples. To obtain an unbiased estimator, we should divide by n–k, where k is the number of parameters, in this case 2. However, the bias disappears as the sample size becomes large.

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section 10.6 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own and who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course 20 Elements of Econometrics