INTERPRETATION OF A REGRESSION EQUATION

INTERPRETATION OF A REGRESSION EQUATION
The scatter diagram shows hourly earnings in 1994 plotted against highest grade completed for a sample of 570 respondents. 1

^ Here is the scatter diagram again, with the regression line shown. 7

^ What do the coefficients actually mean? 8

^ To answer this question, you must refer to the units in which the variables are measured. 9

^ S is measured in years (strictly speaking, grades completed), EARNINGS in dollars per hour. So the slope coefficient implies that hourly earnings increase by $1.07 for each extra year of schooling. 10

^ We will look at a geometrical representation of this interpretation. To do this, we will enlarge the marked section of the scatter diagram. 11

$11.49 $1.07 One year $10.41 The regression line indicates that completing 12th grade instead of 11th grade would increase earnings by $1.073, from $ to $11.486, as a general tendency. 12

^ You should ask yourself whether this is a plausible figure. If it is implausible, this could be a sign that your model is misspecified in some way. 13

^ For low levels of education it might be plausible. But for high levels it would seem to be an underestimate. 14

^ What about the constant term? (Try to answer this question yourself before continuing with this sequence.) 15

^ Literally, the constant indicates that an individual with no years of education would have to pay $1.39 per hour to be allowed to work. 16

^ This does not make any sense at all, an interpretation of negative payment is impossible to sustain. 17

^ A safe solution to the problem is to limit the interpretation to the range of the sample data, and to refuse to extrapolate on the ground that we have no evidence outside the data range. 18

^ With this explanation, the only function of the constant term is to enable you to draw the regression line at the correct height on the scatter diagram. It has no meaning of its own. 19

MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE
EARNINGS = b1 + b2S + b3A+ u Specifically, we will look at an earnings function model where hourly earnings, EARNINGS, depend on years of schooling (highest grade completed), S, and a measure of cognitive ability, A. The model has three dimensions, one each for EARNINGS, S, and A. The starting point for investigating the determination of EARNINGS is the intercept, b1. Literally the intercept gives EARNINGS for those respondents who have no schooling and who scored zero on the ability test. However, the ability score is scaled in such a way as to make it impossible to score zero. Hence a literal interpretation of b1 would be unwise. The next term on the right side of the equation gives the effect of variations in S. A one year increase in S causes EARNINGS to increase by b2 dollars, holding A constant. Similarly, the third term gives the effect of variations in A. A one point increase in A causes earnings to increase by b3 dollars, holding S constant. The final element of the model is the disturbance term, u. In this observation, u happens to have a positive value. 2

A sample consists of a number of observations generated in this way. Note that the interpretation of the model does not depend on whether S and A are correlated or not. However we do assume that the effects of S and A on EARNINGS are additive. The impact of a difference in S on EARNINGS is not affected by the value of A, or vice versa. The regression coefficients are derived using the same least squares principle used in simple regression analysis. The fitted value of Y in observation i depends on our choice of b1, b2, and b3. 11

The residual ei in observation i is the difference between the actual and fitted values of Y. 12

We define RSS, the sum of the squares of the residuals, and choose b1, b2, and b3 so as to minimize it. 13

t TEST OF A HYPOTHESIS RELATING TO A REGRESSION COEFFICIENT
t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … For this reason we need to refer to a table of critical values of t when performing significance tests on the coefficients of a regression equation. 18

t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … At the top of the table are listed possible significance levels for a test. For the time being we will be performing two-tailed tests, so ignore the line for one-tailed tests. 19

t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … Hence if we are performing a (two-tailed) 5% significance test, we should use the column thus indicated in the table. 20

t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … Number of degrees of freedom in a regression = number of observations - number of parameters estimated. The left hand vertical column lists degrees of freedom. The number of degrees of freedom in a regression is defined to be the number of observations minus the number of parameters estimated. 21

t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … In a simple regression, we estimate just two parameters, the constant and the slope coefficient, so the number of degrees of freedom is n - 2 if there are n observations. 22

t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … If we were performing a regression with 20 observations, as in the price inflation/wage inflation example, the number of degrees of freedom would be 18 and the critical value of t for a 5% test would be 23

t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … Note that as the number of degrees of freedom becomes large, the critical value converges on 1.96, the critical value for the normal distribution. This is because the t distribution converges on the normal distribution. 24

t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … If instead we wished to perform a 1% significance test, we would use the column indicated above. Note that as the number of degrees of freedom becomes large, the critical value converges to 2.58, the critical value for the normal distribution. 27

t Distribution: Critical values of t Degrees of Two-tailed test % % % % % % freedom One-tailed test % % % % % % … … … … … … … For a simple regression with 20 observations, the critical value of t at the 1% level is 28

s.d. of b2 known s.d. of b2 not known discrepancy between hypothetical value and sample estimate, in terms of s.d.: discrepancy between hypothetical value and sample estimate, in terms of s.e.: 5% significance test: reject H0: b2 = b2 if z > or z < -1.96 1% significance test: reject H0: b2 = b2 if t > or t < So we should this figure in the test procedure for a 1% test. 29

EARNINGS = b1 + b2 TRAINING + u
EXERCISE 3.10 A researcher with a sample of 50 individuals with similar education but differing amounts of training hypothesizes that hourly earnings, EARNINGS, may be related to hours of training, TRAINING, according to the relationship EARNINGS = b1 + b2 TRAINING + u He is prepared to test the null hypothesis H0: b2 = 0 against the alternative hypothesis H1: b at the 5 percent and 1 percent levels. What should he report 1. If b2 = 0.30, s.e.(b2) = 0.12? 2. If b2 = 0.55, s.e.(b2) = 0.12? 3. If b2 = 0.10, s.e.(b2) = 0.12? 4. If b2 = -0.27, s.e.(b2) = 0.12? 1

n = 50, so 48 degrees of freedom
EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom There are 50 observations and 2 parameters have been estimated, so there are 48 degrees of freedom. 2

n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68
EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 The table giving the critical values of t does not give the values for 48 degrees of freedom. We will use the values for 50 as a guide. For the 5% level the value is 2.01, and for the 1% level it is The critical values for 48 will be slightly higher. 3

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 1. If b2 = 0.30, s.e.(b2) = 0.12? t = 2.50. In the first case, the t statistic is 2.50. 4

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 1. If b2 = 0.30, s.e.(b2) = 0.12? t = Reject H0 at the 5% level but not at the 1% level. This is greater than the critical value of t at the 5% level, but less than the critical value at the 1% level. 5

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 1. If b2 = 0.30, s.e.(b2) = 0.12? t = Reject H0 at the 5%, but not at the 1%, level. In this case we should mention both tests. It is not enough to say "Reject at the 5% level", because it leaves open the possibility that we might be able to reject at the 1% level. 6

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 1. If b2 = 0.30, s.e.(b2) = 0.12? t = Reject H0 at the 5%, but not at the 1%, level. Likewise it is not enough to say "Do not reject at the 1% level", because this does not reveal whether the result is significant at the 5% level or not. 7

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 2. If b2 = 0.55, s.e.(b2) = 0.12? t = 4.58. In the second case, t is equal to 4.58. 8

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 2. If b2 = 0.55, s.e.(b2) = 0.12? t = Reject H0 at the 1% level. We report only the result of the 1% test. There is no need to mention the 5% test. If you do, you reveal that you do not understand that rejection at the 1% level automatically means rejection at the 5% level, and you look ignorant. 9

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 2. If b2 = 0.55, s.e.(b2) = 0.12? t = Reject H0 at the 0.1% level (tcrit, 0.1% = 3.50). Actually, given the large t statistic, it is a good idea to investigate whether we can reject H0 at the 0.1% level. It turns out that we can. The critical value for 50 degrees of freedom is So we just report the outcome of this test. There is no need to mention the 1% test. 10

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 2. If b2 = 0.55, s.e.(b2) = 0.12? t = Reject H0 at the 0.1% level (tcrit, 0.1% = 3.50). Why is it a good idea to press on to a 0.1% test, if the t statistic is large? Try to answer this question before looking at the next slide. 11

EXERCISE 3.10 EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 2. If b2 = 0.55, s.e.(b2) = 0.12? t = Reject H0 at the 0.1% level (tcrit, 0.1% = 3.50). The reason is that rejection at the 1% level still leaves open the possibility of a 1% risk of having made a Type I error (rejecting the null hypothesis when it is in fact true). So there is a 1% risk of the "significant" result having occurred as a matter of chance. 12

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 2. If b2 = 0.55, s.e.(b2) = 0.12? t = Reject H0 at the 0.1% level (tcrit, 0.1% = 3.50). If you can reject at the 0.1% level, you reduce that risk to one tenth of 1%. This means that the result is almost certainly genuine. 13

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 3. If b2 = 0.10, s.e.(b2) = 0.12? t = 0.83. In the third case, t is equal to 0.83. 14

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 3. If b2 = 0.10, s.e.(b2) = 0.12? t = Do not reject H0 at the 5% level. We report only the result of the 5% test. There is no need to mention the 1% test. If you do, you reveal that you do not understand that not rejecting at the 5% level automatically means not rejecting at the 1% level, and you look ignorant. 15

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 4. If b2 = -0.27, s.e.(b2) = 0.12? t = In the fourth case, t is equal to 16

EXERCISE EARNINGS = b1 + b2 TRAINING + u H0: b2 = 0, H1: b2 0 n = 50, so 48 degrees of freedom tcrit, 5% = 2.01, tcrit, 1% = 2.68 _______________________________________________ 4. If b2 = -0.27, s.e.(b2) = 0.12? t = Reject H0 at the 5% level but not at the 1% level. The absolute value of the t statistic is between the critical values for the 5% and 1% tests. So we mention both tests, as in the first case. 17

F TESTS OF GOODNESS OF FIT
This sequence describes two F tests of goodness of fit in a multiple regression model. The first relates to the goodness of fit of the equation as a whole. 1

We will consider the general case where there are k - 1 explanatory variables. For the F test of goodness of fit of the equation as a whole, the null hypothesis, in words, is that the model has no explanatory power at all. 2

Of course we hope to reject it and conclude that the model does have some explanatory power. 3

The model will have no explanatory power if it turns out that Y is unrelated to any of the explanatory variables. Mathematically, therefore, the null hypothesis is that all the coefficients b2, ..., bk are zero. 4

The alternative hypothesis is that at least one of these b coefficients is different from zero. 5

In the multiple regression model there is a difference between the roles of the F and t tests. The F test tests the joint explanatory power of the variables, while the t tests test their explanatory power individually. 6

In the simple regression model the F test was equivalent to the (two-tailed) t test on the slope coefficient because the "group" consisted of just one variable. 7

The F statistic for the test was defined in the last sequence in Chapter 3. ESS is the explained sum of squares and RSS is the residual sum of squares. 8

It can be expressed in terms of R2 by dividing the numerator and denominator by TSS, the total sum of squares. 9

ESS / TSS is equal to R2 and RSS / TSS is equal to (1 - R2). (For proofs, see the last sequence in Chapter 3.) 10

The educational attainment model will be used as an example. We will suppose that S depends on ASVABC, the ability score, and SM, and SF, the highest grade completed by the mother and father of the respondent, respectively. 11

The null hypothesis for the F test of goodness of fit is that all three slope coefficients are equal to zero. The alternative hypothesis is that at least one of them is non-zero. 12

INTERPRETATION OF A REGRESSION EQUATION

Similar presentations

Presentation on theme: "INTERPRETATION OF A REGRESSION EQUATION"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

INTERPRETATION OF A REGRESSION EQUATION

Similar presentations

Presentation on theme: "INTERPRETATION OF A REGRESSION EQUATION"— Presentation transcript:

Similar presentations

About project

Feedback