1. Slope of the regression line:
How to estimate the best fitting line?
y=a+bx+ε
(Observed values of y are contaminated with random errors)
Sum of Squared Errors (SSE) 1 n 1 n
Slope of the regression line:
Correlation coefficient * standard deviation (y) / standard deviation (x)

2. Properties of the estimated regression coefficients
11/11/2018
Properties of the estimated regression coefficients
(1) With increasing sample size the estimated values become closer to the true parameter values of the linear function that relates y with x.
(2) The variance of the observed y-values is the sum of the vaiance ‘explained by the linear model’ and the independent error variance
(3) The correlation coefficient increases as the ratio between explained variance and error variance (signal–to–noise) increases

3. The linear regression estimation problem revisited
Estimate for the regression parameter:
To distinguish the true (but unknown) parameters a, b from the estimated parameters, the ‘hat’ symbol is
often used.

4. The linear regression estimation problem revisited
Estimate for the regression parameter:
Here the overbars denote
the mean of the samples,
sx and sy are the
standard deviations of the
samples of x and y
Slope
In the next slides I assume that the data
are centered and thus the mean of x and y are 0
(and the intercept is of secondary importance
and one can set the intercept to 0)
Intercept

5. The linear regression estimation problem revisited
Estimate for the regression parameter:
Slope

6. The linear regression estimation problem revisited

7. The linear regression estimation problem revisited
Error independent of x
=> Covariance 0 O

8. The linear regression estimation problem revisited
The total variance is the sum
of the variance explained by x
plus the independent error variance )

9. The linear regression estimation problem revisited
The total variance is the sum
of the variance explained by x
plus the independent error variance