Download presentation

Presentation is loading. Please wait.

Published byEden Pease Modified about 1 year ago

1
Part 17: Nonlinear Regression 17-1/26 Econometrics I Professor William Greene Stern School of Business Department of Economics

2
Part 17: Nonlinear Regression 17-2/26 Econometrics I Part 17 – Nonlinear Regression

3
Part 17: Nonlinear Regression 17-3/26 Nonlinear Regression Nonlinearity and Nonlinear Models Estimation Criterion Iterative Algorithm

4
Part 17: Nonlinear Regression 17-4/26 Nonlinear Regression What makes a regression model “nonlinear?” Nonlinear functional form? Regression model: y i = f(x i, ) + i Not necessarily: y i = exp(α) + β 2 *log(x i )+ i β 1 = exp(α) y i = exp(α)x i β 2 exp( i ) is “loglinear” Models can be nonlinear in the functional form of the relationship between y and x, and not be nonlinear for purposes here. We will redefine “nonlinear” shortly, as we proceed.

5
Part 17: Nonlinear Regression 17-5/26 Least Squares Least squares: Minimize wrt S( ) = ½ i {y i - E[y i |x i, ]} 2 = ½ i [y i - f(x i, )] 2 = ½ i e i 2 First order conditions: S( )/ = 0 {½ i [y i - f(x i, )] 2 }/ = ½ i (-2)[y i - f(x i, )] f(x i, )/ = - i e i x i 0 = 0 (familiar?) There is no explicit solution, b = f(data) like LS. (Nonlinearity of the FOC defines nonlinear model)

6
Part 17: Nonlinear Regression 17-6/26 Example How to solve this kind of set of equations: Example, y i = 0 + 1x i 2 + i. [ ½ i e i 2 ]/ 0 = i (-1) (y i - 0 - 1x i 2 ) 1 = 0 [ ½ i e i 2 ]/ 1 = i (-1) (y i - 0 - 1 x i 2 ) x i 2 = 0 [ ½ i e i 2 ]/ 2 = i (-1) (y i - 0 - 1 x i 2 ) 1 x i 2 lnx i = 0 Nonlinear equations require a nonlinear solution. This defines a nonlinear regression model. I.e., when the first order conditions are not linear in . (!!!) Check your understanding. What does this produce if f(x i, ) = x i ? (I.e., a linear model)

7
Part 17: Nonlinear Regression 17-7/26 The Linearized Regression Model Linear Taylor series: y = f(x i, ) + . Expand the regression around some point, *. f(x i, ) f(x i, * ) + k [ f(x i, * )/ k * ]( k - k * ) = f(x i, * ) + k x ik 0 ( k - k * ) = [f(x i, * ) - k x ik 0 k * ] + k x i 0 k = f 0 + k x ik 0 k which looks linear. x ik 0 = the derivative wrt k evaluated at * The ‘pseudo-regressors’ are the derivative functions in the linearized model.

8
Part 17: Nonlinear Regression 17-8/26 Estimating Asy.Var[b] Computing the asymptotic covariance matrix for the nonlinear least squares estimator using the pseudo regressors and the sum of squares.

9
Part 17: Nonlinear Regression 17-9/26 Gauss-Marquardt Algorithm Given a coefficient vector b(m), at step m, find the vector for step m+1 by b(m+1) = b(m) + [X 0 (m)X 0 (m)] -1 X 0 (m)e 0 (m) Columns of X 0 (m) are the derivatives, f(x i,b(m))/ b(m) e 0 = vector of residuals, y - f[x,b(m)] “Update” vector is the slopes in the regression of the residuals on the pseudo-regressors. Update is zero when they are orthogonal. (Just like LS)

10
Part 17: Nonlinear Regression 17-10/26

11
Part 17: Nonlinear Regression 17-11/26

12
Part 17: Nonlinear Regression 17-12/26 The NIST Dan Wood Application Y X y = 1 x 2 + x i 0 = [x 2, 1 x 2 logx ]

13
Part 17: Nonlinear Regression 17-13/26 Dan Wood Solution

14
Part 17: Nonlinear Regression 17-14/26 Iterations NLSQ;LHS=Y ;FCN=B1*X^B2 ;LABELS=B1,B2 ;MAXIT=500;TLF;TLB;OUTPUT=1;DFC ;START=1,5 $ Begin NLSQ iterations. Linearized regression. Iteration= 1; Sum of squares= ; Gradient = Iteration= 2; Sum of squares= ; Gradient = Iteration= 3; Sum of squares= E-01; Gradient = E-01 Iteration= 4; Sum of squares= E-02; Gradient = E-05 Iteration= 5; Sum of squares= E-02; Gradient = E-11 Iteration= 6; Sum of squares= E-02; Gradient = E-15 Iteration= 7; Sum of squares= E-02; Gradient = E-19 Iteration= 8; Sum of squares= E-02; Gradient = E-23 Convergence achieved

15
Part 17: Nonlinear Regression 17-15/26 Results User Defined Optimization Nonlinear least squares regression LHS=Y Mean = Standard deviation = Number of observs. = 6 Model size Parameters = 2 Degrees of freedom = 4 Residuals Sum of squares = E-02 Standard error of e = Fit R-squared = Adjusted R-squared = Model test F[ 1, 4] (prob) = (.0000) Not using OLS or no constant. Rsqrd & F may be < | Standard Prob. 95% Confidence UserFunc| Coefficient Error z |z|>Z* Interval B1|.76886*** B2| *** Note: ***, **, * ==> Significance at 1%, 5%, 10% level

16
Part 17: Nonlinear Regression 17-16/26 NLS Solution The pseudo regressors and residuals at the solution are: X10 X20 X30 1 x 2 1 x 2 lnx e X0e0 = D D D-10

17
Part 17: Nonlinear Regression 17-17/26 Application: Doctor Visits German Individual Health Care data: N=27,236 Model for number of visits to the doctor

18
Part 17: Nonlinear Regression 17-18/26 Conditional Mean and Projection Notice the problem with the linear approach. Negative predictions.

19
Part 17: Nonlinear Regression 17-19/26 Nonlinear Model Specification Nonlinear Regression Model y=exp(β’x) + ε X =one,age,health_status, married, education, household_income, kids

20
Part 17: Nonlinear Regression 17-20/26 NLS Iterations --> nlsq;lhs=docvis;start=0,0,0,0,0,0,0;labels=k_b;fcn=exp(b1'x);maxit=25;out... Begin NLSQ iterations. Linearized regression. Iteration= 1; Sum of squares= ; Gradient= Iteration= 2; Sum of squares= E+11 ; Gradient= E+11 Iteration= 3; Sum of squares= E+10 ; Gradient= E+10 Iteration= 4; Sum of squares= ; Gradient= Iteration= 5; Sum of squares= ; Gradient= Iteration= 6; Sum of squares= ; Gradient= Iteration= 7; Sum of squares= ; Gradient= Iteration= 8; Sum of squares= ; Gradient= Iteration= 9; Sum of squares= ; Gradient= Iteration= 10; Sum of squares= ; Gradient= Iteration= 11; Sum of squares= ; Gradient= E-03 Iteration= 12; Sum of squares= ; Gradient= E-05 Iteration= 13; Sum of squares= ; Gradient= E-07 Iteration= 14; Sum of squares= ; Gradient= E-08 Iteration= 15; Sum of squares= ; Gradient= E-10 (The blue iterations take place within the ‘hidden digits.’)

21
Part 17: Nonlinear Regression 17-21/26 Nonlinear Regression Results

22
Part 17: Nonlinear Regression 17-22/26 Partial Effects in the Nonlinear Model

23
Part 17: Nonlinear Regression 17-23/26 Delta Method for Asymptotic Variance of the Slope Estimator

24
Part 17: Nonlinear Regression 17-24/26 Computing the Slopes Namelist;x=one,age,hsat,married,educ,hhninc,hhkids$ Calc ; k=col(x)$ Nlsq ; lhs=docvis;start=0,0,0,0,0,0,0 ; labels=k_b;fcn=exp(b1'x)$ Partials ; function=exp(b1'x) ; parameters=b ; covariance = varb ; labels=k_b ; effects : x ; summary [; means] $

25
Part 17: Nonlinear Regression 17-25/26 Partial Effects: PEA vs. APE

26
Part 17: Nonlinear Regression 17-26/26 What About Just Using LS? |Variable | Coefficient | Standard Error |b/St.Er.|P[|Z|>z] | Mean of X| Least Squares Coefficient Estimates AGE NEWHSAT MARRIED EDUC HHNINC HHKIDS Estimated Partial Effects ME_ ME_ ME_ ME_ ME_ ME_

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google