Download presentation

Presentation is loading. Please wait.

1
The Multiple Regression Model

2
**Two Explanatory Variables**

yt = 1 + 2xt2 + 3xt3 + εt xt affect yt separately yt xt2 = 2 xt3 yt = 3 But least squares estimation of 2 now depends upon both xt2 and xt3 .

3
**Correlated Variables yt = 1 + 2xt2 + 3xt3 + εt yt = output**

xt2 = capital xt3 = labor Always 5 workers per machine. If number of workers per machine is never varied, it becomes impossible to tell if the machines or the workers are responsible for changes in output.

4
**The General Model yt = 1 + 2xt2 + 3xt3 +. . .+ KxtK + εt**

The parameter 1 is the intercept (constant) term. The variable attached to 1 is xt1= 1. Usually, the number of explanatory variables is said to be K1 (ignoring xt1= 1), while the number of parameters is K. (Namely: K).

5
**Statistical Properties of εt**

1. E(εt) = 0 2. var(εt) = 2 covεt , εs= for t s 4. εt ~ N(0, 2)

6
**Statistical Properties of yt**

1. E (yt) = 1 + 2xt KxtK 2. var(yt) = var(εt) = 2 cov(yt ,ys) = cov(εt , εs)= 0 t s 4. yt ~ N(1+2xt KxtK, 2)

7
**Assumptions 1. yt = 1 + 2xt2 +. . .+ KxtK + εt**

2. E (yt) = 1 + 2xt KxtK 3. var(yt) = var(εt) = 2 cov(yt ,ys) = cov(εt , εs) = 0 t s 5. The values of xtk are not random 6. yt ~ N(1+2xt KxtK, 2)

8
**Least Squares Estimation**

yt = 1 + 2xt2 + 3xt3 + εt T S S(1, 2, 3) = yt12xt23xt3 t=1 yt = yt y * Define: xt2 = xt2 x2 * xt3 = xt3 x3 *

9
**Least Squares Estimators**

b1 = y – b2x2 – b3x3 b2 = yt xt2xt3 yt xt3xt2xt3 * 2 xt2 xt3 xt2xt3 b3 = yt xt3xt2 yt xt2xt3xt2 * 2 xt2 xt3 xt2xt3

10
**Dangers of Extrapolation**

Statistical models generally are good only within the relevant range. This means that extending them to extreme data values outside the range of the original data often leads to poor and sometimes ridiculous results. If height is normally distributed and the normal ranges from minus infinity to plus infinity, pity the man minus three feet tall.

11
**Interpretation of Coefficients**

bj represents an estimate of the mean change in y responding to a one-unit change in xj when all other independent variables are held constant. Hence, bj is called the partial coefficient. Note that regression analysis cannot be interpreted as a procedure for establishing a cause-and-effect relationship between variables.

12
Universal Set B x3 / x2 x2 / x3 x2 x3

13
**Error Variance Estimation**

Unbiased estimator of the error variance: 2 ^ = εt Transform to a chi-square distribution: 2 2 2 ^

14
Gauss-Markov Theorem Under the first of five assumptions of the multiple regression model, the ordinary least squares estimators have the smallest variance of all linear and unbiased estimators. This means that the least squares estimators are the Best Linear U nbiased Estimators (BLUE).

15
**(xt2 x2)(xt3 x3) (xt2 x2)2 (xt3 x3)2 Variances 2 2**

yt = 1 + 2xt2 + 3xt3 + εt var(b2) = (1 r23)(xt2 x2)2 2 2 When r23 = 0 these reduce to the simple regression formulas. var(b3) = (1 r23)(xt3 x3)2 2 2 (xt2 x2)2 (xt3 x3)2 where r23 = (xt2 x2)(xt3 x3)

16
**Variance Decomposition**

The variance of an estimator is smaller when: 1. The error variance, 2, is smaller: 2. The sample size, T, is larger: (xt2 x2)2 . 3. The variable values are more spread out: (xt2 x2)2 . 4. The correlation is close to zero: r 2 t = 1 T

17
**(xt2 x2)(xt3 x3) (xt2 x2)2 (xt3 x3)2 Covariances**

yt = 1 + 2xt2 + 3xt3 + εt (1 r23) (xt2 x2)2 (xt3 x3)2 cov(b2,b3) = 2 r23 2 where r23 = (xt2 x2)2 (xt3 x3)2 (xt2 x2)(xt3 x3)

18
**Covariance Decomposition**

The covariance between any two estimators is larger in absolute value when: 1. The error variance, 2, is larger. 2. The sample size, T, is smaller. 3. The values of the variables are less spread out. 4. The correlation, r23, is high.

19
**Var-Cov Matrix yt = 1 + 2xt2 + 3xt3 + εt**

The least squares estimators b1, b2, and b3 have covariance matrix: var(b1) cov(b1,b2) cov(b1,b3) cov(b1,b2,b3) = cov(b1,b2) var(b2) cov(b2,b3) cov(b1,b3) cov(b2,b3) var(b3)

20
**Normal yt = 1 + 2x2t + 3x3t +. . .+ KxKt + εt**

yt ~N (1 + 2x2t + 3x3t KxKt), 2 εt ~ N(0, 2) This implies and is implied by: Since bk is a linear function of the yt: bk ~ N k, var(bk) z = ~ N(0,1) for k = 1,2,...,K bk k var(bk)

21
**Student-t ^ ^ t has a Student-t distribution with df=(TK).**

Since generally the population variance of bk , var(bk) , is unknown, we estimate it with which uses 2 instead of 2. var(bk) ^ bk k var(bk) ^ t = = se(bk) t has a Student-t distribution with df=(TK).

22
**Interval Estimation tc is critical value for (T-K) degrees of freedom**

bk k se(bk) P tc < < tc = 1 tc is critical value for (T-K) degrees of freedom such that P( t > tc ) = /2. P bk tc se(bk) < k < bk + tc se(bk) = 1 bk tc se(bk) , bk + tc se(bk) Interval endpoints:

23
**Student - t Test yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt H0: 1 = 0**

Student-t tests can be used to test any linear combination of the regression coefficients: H0: 1 = 0 H0: 2 + 3 + 4 = 1 H0: 32 73 = 21 H0: 2 3 < 5 Every such t-test has exactly TK degrees of freedom where K = # of coefficients estimated(including the intercept).

24
**One Tail Test yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt H0: 3 < 0 b3**

se(b3) t = ~ t (TK) df = TK = T4 tc

25
**Two Tail Test yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt H0: 2 = 0 b2**

se(b2) t = ~ t (TK) df = TK = T4 -tc tc

26
**(yt y)2 R2 = = 0 < R2 < 1 Goodness - of - Fit ^ SSR SST**

Coefficient of Determination SST R2 = = (yt y)2 t = 1 T ^ SSR 0 < R2 < 1

27
**R2 = = 1 R2 = 1 Adjusted R-Squared SSR SSE SST SSE/(TK) SST/(T1)**

Adjusted Coefficient of Determination Original: SST = 1 SSE R2 = SSR Adjusted: SST/(T1) R2 = 1 SSE/(TK)

28
**Computer Output b2 t = = se(b2) 6.642 2.081 3.191**

Table Summary of Least Squares Results Variable Coefficient Std Error t-value p-value constant price advertising b2 se(b2) t = = 6.642 3.191 2.081

29
**Reporting Your Results**

Reporting standard errors: ^ yt = Xt2 + Xt3 (6.48) (3.191) (0.167) (s.e.) Reporting t-statistics: ^ yt = Xt2 + Xt3 (16.17) (-2.081) (17.868) (t)

30
yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt H0: 2 = 0 H1: 2 = 0 H0: yt = 3Xt3 + 4Xt4 + εt H1: yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt H0: Restricted Model H1: Unrestricted Model

31
**Single Restriction F-Test**

yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt H0: 2 = 0 H1: 2 = 0 Under H0 (SSER SSEU)/J SSEU/(TK) F = ~ FJ, T-K ( )/1 /(52 3) = dfn = J = 1 = dfd = TK = 49 By definition this is the t-statistic squared: t = 2.081 F = t2 =

32
yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt H0: 2 = 0, 4 = 0 H1: H0 not true H0: yt = 3Xt εt H1: yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt H0: Restricted Model H1: Unrestricted Model

33
**Multiple Restriction F-Test**

yt = 1 + 2Xt2 + 3Xt3 + 4Xt4 + εt (SSER SSEU)/J SSEU/(TK) F = H0: 2 = 0, 4 = 0 H1: H0 not true Under H0 ~ F J, T-K First run the restricted regression by dropping Xt2 and Xt4 to get SSER. dfn = J = 2 (J: The number of hypothesis) dfd = TK = 49 Next run unrestricted regression to get SSEU .

34
**F-Tests f(F) F = ~ F J, T-K F**

F-Tests of this type are always right-tailed, even for left-sided or two-sided hypotheses, because any deviation from the null will make the F value bigger (move rightward). f(F) (SSER SSEU)/J SSEU/(TK) F = ~ F J, T-K Fc F

35
**F-Test of Entire Equation**

yt = 1 + 2Xt2 + 3Xt3 + εt We ignore 1. Why? H0: 2 = 3 = 0 H1: H0 not true (SSER SSEU)/J SSEU/(TK) F = dfn = J = 2 ( )/2 /(52 3) = dfd = TK = 49 = 0.05 = Reject H0! F2, 49, = 3.187

36
**R2 = = ANOVA Table SSR SST 0.867 Table 8.3 Analysis of Variance Table**

Sum of Mean Source DF Squares Square F-Value Regression Error Total p-value: SST R2 = = SSR 0.867

37
**Nonsample Information**

A certain production process is known to be Cobb-Douglas with constant returns to scale. ln(yt) = 1 + 2 ln(Xt2) + 3 ln(Xt3) + 4 ln(Xt4) + εt where 2 + 3 + 4 = 1 4 = (1 2 3) ln(yt /Xt4) = 1 + 2 ln(Xt2/Xt4) + 3 ln(Xt3 /Xt4) + εt yt = 1 + 2 Xt2 + 3 Xt3 + εt * Run least squares on the transformed model. Interpret coefficients same as in original model.

38
**Collinear Variables The term independent variables means**

an explanatory variable is independent of of the error term, but not necessarily independent of other explanatory variables. Since economists typically have no control over the implicit experimental design, explanatory variables tend to move together which often makes sorting out their separate influences rather problematic.

39
**Effects of Collinearity**

A high degree of collinearity will produce: 1. no least squares output when collinearity is exact. 2. large standard errors and wide confidence intervals. 3. insignificant t-values even with high R2 and a significant F-value. 4. estimates sensitive to deletion or addition of a few observations or insignificant variables. 5.The OLS estimators retain all their desired properties (BLUE and consistency), but the problem is that the influential procedure may be uninformative.

40
**Identifying Collinearity**

Evidence of high collinearity include: 1. a high pairwise correlation between two explanatory variables (greater than .8 or .9). a high R-squared (called Rj2) when regressing one explanatory variable (Xj) on the other explanatory variables. Variance inflation factor (VIF): VIF (bj) = 1 / (1 Rj2) ( > 10) 3. high R2 and a statistically significant F-value when the t-values are statistically insignificant.

41
**Mitigating Collinearity**

High collinearity is not a violation of any least squares assumption, but rather a lack of adequate information in the sample: 1. Collect more data with better information. 2. Impose economic restrictions as appropriate. Impose statistical restrictions when justified. Delete the variable which is highly collinear with other explanatory variables.

42
**Prediction yt = 1 + 2Xt2 + 3Xt3 + εt y0 = b1 + b2X02 + b3X03**

Given a set of values for the explanatory variables, (1 X02 X03), the best linear unbiased predictor of y is given by: y0 = b1 + b2X02 + b3X03 ^ This predictor is unbiased in the sense that the average value of the forecast error is zero.

Similar presentations

Presentation is loading. Please wait....

OK

SIMPLE LINEAR REGRESSION

SIMPLE LINEAR REGRESSION

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on precautions of tsunami warnings Ppt on question tags youtube Ppt on synthesis and degradation of purines and pyrimidines dna Ppt on creativity and innovation Ppt on cloud computing challenges Ppt on coalition government in india Ppt on intelligent manufacturing in industrial automation Ppt on articles in grammar Ppt on stars and solar system for class 8 Ppt on channels of distribution activities