Undergraduated Econometrics Chapter 6 The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 1

Undergraduated Econometrics Chapter Contents 6.1 The Coefficient of Determination 6.2 Reporting Regression Results 6.3 Choosing a Functional Form 6.4 Are the Residuals Normally Distributed Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 2

The Coefficient of Determination 6.1 The Coefficient of Determination Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 3

There are two major reasons for analyzing the model 6.1 The Coefficient of Determination There are two major reasons for analyzing the model （6.1.1） to explain how the dependent variable (yi) changes as the independent variable (xi) changes to predict y0 given an x0 Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4

6.1 The Coefficient of Determination Closely allied with the prediction problem is the desire to use xi to explain as much of the variation in the dependent variable yi as possible. In the regression model Eq. 6.1.1 we call xi the ‘‘explanatory’’ variable because we hope that its variation will ‘‘explain’’ the variation in yi Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 5

E(yi) is the explainable or systematic part 6.1 The Coefficient of Determination To develop a measure of the variation in yi that is explained by the model, we begin by separating yi into its explainable and unexplainable components. E(yi) is the explainable or systematic part ei is the random, unsystematic and unexplainable component Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 6

Analogous to Eq. 4.8, we can write: （6.1.3） The Coefficient of Determination Analogous to Eq. 4.8, we can write: （6.1.3） Subtracting the sample mean from both sides: （6.1.4） Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 7

Figure 6.1 Explained and unexplained components of yi The Coefficient of Determination Figure 6.1 Explained and unexplained components of yi Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 8 Page 4

6.1 The Coefficient of Determination Squaring and summing both sides of Eq. 6.1.4, and using the fact that we get: Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 9

These are called ‘‘sums of squares’’ 6.1 The Coefficient of Determination Eq. 6.1.5 decomposition of the ‘‘total sample variation’’ in y into explained and unexplained components These are called ‘‘sums of squares’’ Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 10

Specifically: 6.1 The Coefficient of Determination Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 11

Thus Eq. 6.1.5 becomes: 6.1 The Coefficient of Determination Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 12

6.1 The Coefficient of Determination Let’s define the coefficient of determination, or R2 , as the proportion of variation in y explained by x within the regression model: Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 13

6.1 The Coefficient of Determination We can see that: The closer R2 is to 1, the closer the sample values yi are to the fitted regression equation If R2 = 1, then all the sample data fall exactly on the fitted least squares line, so SSE = 0, and the model fits the data ‘‘perfectly’’ If the sample data for y and x are uncorrelated and show no linear association, then the least squares fitted line is ‘‘horizontal,’’ and identical to y, so that SSR = 0 and R2 = 0 Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 14

6.1 The Coefficient of Determination When 0 < R2 < 1 then R2 is interpreted as ‘‘the proportion of the variation in y about its mean that is explained by the regression model’’ Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 15

For the food expenditure example, the sums of squares are: 6.1 The Coefficient of Determination 6.1.1 Analysis of Variance Table and R2 for Food Expenditure Example For the food expenditure example, the sums of squares are: Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 16

6.1 The Coefficient of Determination 6.1.1 Analysis of Variance Table and R2 for Food Expenditure Example Therefore: We conclude that 31.7% of the variation in food expenditure (about its sample mean) is explained by our regression model, which uses only income as an explanatory variable Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 17

The correlation coefficient ρxy between x and y is defined as: 6.1 The Coefficient of Determination 6.1.2 Correlation Analysis The correlation coefficient ρxy between x and y is defined as: Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 18

r= cov ^ (𝑋,𝑌) var ^ (𝑋) var ^ (𝑌) (6.1.9) The Coefficient of Determination 6.1.2 Correlation Analysis Given a sample of data pairs (xt, yt), t=1,…, T, the sample correlation coefficient is obtained by replacing the covariance and variances in (6.1.8) by their sample analogues: r= cov ^ (𝑋,𝑌) var ^ (𝑋) var ^ (𝑌) (6.1.9) Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 19

Where So we can write r as 6.1 The Coefficient of Determination 6.1.2 Correlation Analysis Where So we can write r as Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 20

Two relationships between R2 and rxy: r2xy = R2 6.1 The Coefficient of Determination 6.1.3 Correlation Analysis and R2 Two relationships between R2 and rxy: r2xy = R2 R2 can also be computed as the square of the sample correlation coefficient between yi and Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 21

Reporting Regression Results 6.2 Reporting Regression Results Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 22

The key ingredients in a report are: the coefficient estimates 6.2 Reporting Regression Results The key ingredients in a report are: the coefficient estimates the standard errors (or t-values) an indication of statistical significance R2 Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 23

6.2 Reporting Regression Results One way to summarize the regression results is in the form of a “fitted” regression equation: Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 24

Possible effects of scaling the data: 6.2 Reporting Regression Results 6.2.1 The Effects of Scaling the Data Possible effects of scaling the data: Changing the scale of x: the coefficient of x must be multiplied by c, the scaling factor When the scale of x is altered, the only other change occurs in the standard error of the regression coefficient, but it changes by the same multiplicative factor as the coefficient, so that their ratio, the t-statistic, is unaffected All other regression statistics are unchanged Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 26

Possible effects of scaling the data (Continued): 6.2 Reporting Regression Results 6.2.1 The Effects of Scaling the Data Possible effects of scaling the data (Continued): Changing the scale of y: If we change the units of measurement of y, but not x, then all the coefficients must change in order for the equation to remain valid Because the error term is scaled in this process the least squares residuals will also be scaled This will affect the standard errors of the regression coefficients, but it will not affect t-statistics or R2 Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 27

Possible effects of scaling the data (Continued): 6.2 Reporting Regression Results 6.2.1 The Effects of Scaling the Data Possible effects of scaling the data (Continued): Changing the scale of y and x by the same factor: there will be no change in the reported regression results for b2 , but the estimated intercept and residuals will change t-statistics and R2 are unaffected. The interpretation of the parameters is made relative to the new units of measurement. Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 27

Choosing a Function Form 6.3 Choosing a Function Form Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 28

6.3 Choosing a Function Form By “linear” in the parameters” we mean that the parameters are not multiplied together, divided, squared, cubed, and so on. The variables, however, can be transformed in any convenient way, as long as the resulting model satisfies assumptions SR1-SR5 of the simple linear regression model. Economic theory does not always imply that there is a linear relationship. Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 29

6.3 Choosing a Function Form Figure 6.2 A nonlinear relationship between food expenditure and income Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 30 Page 4

Reciprocal: if x is a variable, then its reciprocal is 1/x 6.3 Choosing a Function Form 6.3.1 Some commonly used Functional Forms Choosing an algebraic form for the relationship means choosing transformations of the original variables. The variable transformations that we begin with are: Natural logarithm: If x is a variable, then its natural logarithm is ln(x) Reciprocal: if x is a variable, then its reciprocal is 1/x Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 31

Figure 6.3 Alternative functional forms Choosing a Function Form Figure 6.3 Alternative functional forms 6.3.1 Some commonly used Functional Forms Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 32 Page 4

Summary of three configurations: 6.3 Choosing a Function Form 6.3.1 Some commonly used Functional Forms Summary of three configurations: The model that is linear in the variables describes fitting a straight line to the original data, with slope β2 and point elasticity β2xt/yt The slope of the relationship is constant, but the elasticity changes at each point. The reciprocal model takes shapes shown in Figure 6.3a. As x increase, y approaches the intercept, its asymptote, from above or below depending on the sign of β2. Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 33

Summary of three configurations (continue): 6.3 Choosing a Function Form 6.3.1 Some commonly used Functional Forms Summary of three configurations (continue): 3. In the log-log model all values of y and x must be positive. The slopes of these curves change at every point, but the elasticity is constant and equal to β2. 4. In the log-linear model only the dependent variable is transformed by the logarithm. 5. In the linear-log model the variable x is transformed by the natural logarithm. Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 34

The reciprocal model is 6.3 Choosing a Function Form 6.3.2 Examples Using Alternative Functional Forms Suppose that in the food expenditure model we wish to choose a functional form that is consistent with figure 6.2 . The reciprocal model is The linear-log model is Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 35

The reciprocal model is 6.3 Choosing a Function Form 6.3.2 Examples Using Alternative Functional Forms Suppose that in the food expenditure model we wish to choose a functional form that is consistent with figure 6.2 . The reciprocal model is The linear-log model is These results are consistent with the idea that a high incomes, and large food expenditures, the effect of an increase in income on food expenditure is small. Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 36

Some other economic models and functional forms 6.3 Choosing a Function Form 6.3.2 Examples Using Alternative Functional Forms Some other economic models and functional forms 1. Demand models: statistical models of the relationship between quantity demanded (yd) and price (x) very frequently taken to be linear in the variables. 2. Supply models: if ys is the quantity supplies, then its relationship to price is often assumed to be linear, creating a linear supply curve. 3. Production functions: another basic economic relationship that can be investigated using the simple linear statistical model is a production function that relates the output of a good produced to the amount of a variable input. Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 37

Some other economic models and functional forms (continued) 6.3 Choosing a Function Form 6.3.2 Examples Using Alternative Functional Forms Some other economic models and functional forms (continued) 4. Cost functions: a family of cost curves, which can be estimated using the simple linear regression model, is based on a “quadratic” total cost curve. 5. The Phillips curve: this important relationship conjectures a systematic relationship between changes in the wage rate and changes in the level of unemployment. Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 38

Figure 6.4 Scatter plot of wheat yield over time 6.3 Choosing a Function Form Figure 6.4 Scatter plot of wheat yield over time 6.3.3 Empirical Issues Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 38 Page 4

One problem with the linear equation 6.3 Choosing a Function Form 6.3.3 Empirical Issues One problem with the linear equation is that it implies that yield increases at the same constant rate β2, when, from Figure 6.4, we expect this rate to be increasing The least squares estimated equation is: Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 40

Figure 6.5 Predicted, actual and residual values from straight line 6.3 Choosing a Function Form Figure 6.5 Predicted, actual and residual values from straight line 6.3.3 Empirical Issues Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 41 Page 4

Perhaps a better model would be: 6.3 Choosing a Function Form 6.3.3 Empirical Issues Perhaps a better model would be: But note that the values of x t3 can get very large This variable is a good candidate for scaling. Define zt = x t3/1000000 The least squares fitted line is: Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 42

Figure 6.6 Bar chart of residuals from straight line 6.3 Choosing a Function Form Figure 6.6 Bar chart of residuals from straight line 6.3.3 Empirical Issues Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 43 Page 4

Are the Residuals Normally Distributed? 6.4 Are the Residuals Normally Distributed? Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 44

Are they normally distributed? 6.4 Are the Residuals Normally Distributed? Hypothesis tests and interval estimates for the coefficients rely on the assumption that the errors, and hence the dependent variable y, are normally distributed Are they normally distributed? Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 45

We can check the distribution of the residuals using: A histogram 6.4 Are the Residuals Normally Distributed? We can check the distribution of the residuals using: A histogram Formal statistical test Merely checking a histogram is not a formal test Many formal tests are available A good one is the Jarque–Bera test for normality Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 46

6.4 Are the Residuals Normally Distributed? Figure 6.8 EViews output: residuals histogram and summary statistics for food expenditure Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 47 Page 4

Skewness refers to how symmetric the residuals are around zero 6.4 Are the Residuals Normally Distributed? The Jarque–Bera test for normality is based on two measures, skewness and kurtosis Skewness refers to how symmetric the residuals are around zero Perfectly symmetric residuals will have a skewness of zero The skewness value for the food expenditure residuals is 0.397 Kurtosis refers to the ‘‘peakedness’’ of the distribution. For a normal distribution the kurtosis value is 3 Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 48

The Jarque–Bera statistic is given by: 6.4 Are the Residuals Normally Distributed? The Jarque–Bera statistic is given by: where N = sample size S = skewness K = kurtosis Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 49

6.4 Are the Residuals Normally Distributed? When the residuals are normally distributed, the Jarque–Bera statistic has a chi-squared distribution with 2 degrees of freedom We reject the hypothesis of normally distributed errors if a calculated value of the statistic exceeds a critical value selected from the chi-squared distribution with two degrees of freedom The 5% critical value from a χ2-distribution with 2 degrees of freedom is 5.99. Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 50

For the food expenditure example, the Jarque–Bera statistic is: 6.4 Are the Residuals Normally Distributed? For the food expenditure example, the Jarque–Bera statistic is: Because 1.077 < 5.99 there is insufficient evidence from the residuals to conclude that the normal distribution assumption is unreasonable at the 5% level of significance Undergraduated Econometrics Chapter 6: The Simple Linear Regression Model: Reporting the Results and choosing the Functional Form Page 4 Page 51