1. FIN357 Li1 The Simple Regression Model y =  0 +  1 x + u

2. FIN357 Li2 Some Terminology In the simple linear regression model, where y =  0 +  1 x + u, we typically refer to y as the Dependent Variable, or Left-Hand Side Variable, or Explained Variable, or

3. FIN357 Li3 Some Terminology we typically refer to x as the Independent Variable, or Right-Hand Side Variable, or Explanatory Variable, or Regressor, or Control Variables

4. FIN357 Li4 A Simple Assumption The average value of u, the error term, in the population is 0. That is, E(u) = 0

5. FIN357 Li5 We also assume E(u|x) = 0 E(y|x) =  0 +  1 x

6. FIN357 Li6.. x1x1 x2x2 E(y|x) as a linear function of x, where for any x the distribution of y is centered about E(y|x) E(y|x) =  0 +  1 x y f(y)

7. FIN357 Li7 Ordinary Least Squares (OLS) Let {(x i,y i ): i=1, …,n} denote a random sample of size n from the population For each observation in this sample, it will be the case that y i =  0 +  1 x i + u i

8. FIN357 Li8.... y4y4 y1y1 y2y2 y3y3 x1x1 x2x2 x3x3 x4x4 } } { { u1u1 u2u2 u3u3 u4u4 x y Population regression line, sample data points and the associated error terms E(y|x) =  0 +  1 x

9. FIN357 Li9 Basic idea of regression is to estimate the population parameters from a sample Intuitively, OLS is fitting a line through the sample points such that the sum of squared residuals is as small as possible. The residual, û, is an estimate of the error term, u, and is the difference between the fitted line (sample regression function) and the sample point

10. FIN357 Li10.... y4y4 y1y1 y2y2 y3y3 x1x1 x2x2 x3x3 x4x4 } } { { û1û1 û2û2 û3û3 û4û4 x y Sample regression line, sample data points and the associated estimated error terms (residuals)

11. FIN357 Li11 One approach to estimate coefficients Given the intuitive idea of fitting a line, we can set up a formal minimization problem That is, we want to choose our parameters such that we minimize the following:

12. FIN357 Li12 It could be shown that estimated coefficient is

13. FIN357 Li13 Summary of OLS slope estimate The slope estimate is the sample covariance between x and y divided by the sample variance of x If x and y are positively correlated, the slope will be positive If x and y are negatively correlated, the slope will be negative

14. FIN357 Li14 Algebraic Properties of OLS: in English The sum of the OLS residuals is zero Thus, the sample average of the OLS residuals is zero as well The sample covariance between the regressors and the OLS residuals is zero The OLS regression line always goes through the mean of the sample

15. FIN357 Li15 Algebraic Properties of OLS: In mathematics:

16. FIN357 Li16 More terminology

17. FIN357 Li17 Notations Alert The notation SSR (Sum of Squared Residuals) in this handout and my other lecture slides= ESS (Error Sum of Squares) in our textbook. The notation SSE (Sum of Squared Explained) in this handout and my other lecture slides = RSS (Regressed Sum of Squares) in our textbook.

18. FIN357 Li18 Goodness-of-Fit How do we think about how well our sample regression line fits our sample data? Can compute the fraction of the total sum of squares (SST) that is explained by the model, call this the R-squared of regression R 2 = SSE/SST = 1 – SSR/SST

19. FIN357 Li19 OLS regressions Now that we’ve derived the formula for calculating the OLS estimates of our parameters, you’ll be happy to know you don’t have to compute them by hand Regressions in GRETL are very simple. Have you installed the software yet?

20. FIN357 Li20 Under some conditions, OLS esimated coefficients are unbiased.

21. FIN357 Li21 Unbiasedness Summary The OLS estimates of  1 and  0 are unbiased Remember unbiasedness is a description of the estimator – in a given sample we may be “near” or “far” from the true parameter

22. FIN357 Li22 Variance of the OLS Estimators Now we know that the sampling distribution of our estimated coefficient is centered around the true parameter Want to think about how spread out this distribution is Assume Var(u|x) =Var(u) =  2

23. FIN357 Li23 Variance of OLS estimators  2 is called the error variance , the square root of the error variance is called the standard deviation of the error E(y|x)=  0 +  1 x and Var(y|x) =  2

24. FIN357 Li24 Variance of OLS estimator

25. FIN357 Li25 Variance of OLS Summary The larger the error variance,  2, the larger the variance of the slope estimate The larger the variability in the x i, the smaller the variance of the slope estimate Problem that the error variance is unknown

26. FIN357 Li26 Estimating the Error Variance We don’t know what the error variance,  2, is, because we don’t observe the errors, u i What we observe are the residuals, û i We can use the residuals to form an estimate of the error variance

27. FIN357 Li27 Estimating the Error Variance

28. FIN357 Li28 Estimating Standard Error of coefficients Estimate