1. Tutorial 6 SEG 7550
23rd Oct.

2. Outline Suggested solution of assignment 1
Introduction to regression analysis
Implement regression in MATLAB

3. Suggested solution of assignment 1
Suggested solution to assignment 1.pdf on website.

4. Introduction to regression analysis
(1) Total Sum of Squares (SST)/Sum of Squares Error(SSE)/ Sum of Squared Residuals (SSR)
Suppose the (x,y) data can be written as follows:
where u is the residual and x is the regressor.
There are two important assumptions for the OLS estimation.

5. Introduction to regression analysis
Two assumptions
a. The expected value of residual is zero
b. The covariance between x and u is zero.

6. Introduction to regression analysis
Based on this properties, we can further derive SST/SSE/SSR and their relationship

7. Introduction to regression analysis
Relationship of SST, SSE and SSR:
proof

8. Introduction to regression analysis
In the proof, why does this equation hold?

9. Introduction to regression analysis
R-Squared/Coefficient of Determination
Goodness of Fit (familiar?) is somehow measure by the following R-squared. R-squared is the ratio of the explained variation compared to the total variation. It is interpreted as the fraction of the sample variation in y that is explained by x.
R-Squared is 1 when it is a prefect fit. If R-Squared is 0, it indicates the poor fit

10. Introduction to regression analysis
Degree of Freedom (df)
The degrees of freedom for the general OLS problem with n observations and k independent variables.
df = n - (k+1)
as there are k+1 parameters in a regression model for k independent variables and an intercept (i.e. constant term).

11. Introduction to regression analysis
Standard Error of Regression
Please note that u_hat_i is different from u_i.
The expected value:

12. Introduction to regression analysis
Standard Error of Regression
Its variance should be:
However, this estimation is bias as it doesn't account for the two restrictions that must be satisfied by the OLS residuals:

13. Introduction to regression analysis
Standard Error of Regression
If there are k-1 independent variables, the degrees of freedom are n- k. Consequently, the unbiased estimator (or the standard error) is

14. Introduction to regression analysis
Testing Hypotheses
Generally, the population model can be written as 
Suppose it satisfies the Linear Regression model. We can set up a simple hypothesis. In most applications, the primary interest is only focused on the null hypothesis

15. Introduction to regression analysis
Testing Hypotheses
If it is true, it implies that the value of y doesn't depend on x_i. Therefore, we can use the t statistic, which is defined by
where s is the standard deviation of sample, Miu is the sample mean and k is the value in null hypotheses

16. Introduction to regression analysis
Testing Hypotheses
F test
The t statistic associated with any OLS coefficient can be used to test whether the corresponding unknown parameter in the population is equal to any given constant (which is usually, but not always zero). If we wish to test multiple hypotheses about the underlying parameters beta_0, beta_1, ..., beta_{k-1}, we will look at the F statistic. Most regression packages consider the overall significance of a regression. It sets the null hypothesis for k-1 independent variables as:
H_0 : x_1, x_2, ..., x_{k-1} do not help to explain y
H_1 : H_0 is not true

17. Introduction to regression analysis
Testing Hypotheses
The F statistic for testing can be written as:
where
The p-values of F Test are especially useful. A small p-value is evidence against H_0.

18. Introduction to regression analysis
Testing Hypotheses
Durbin Watson
A test for AR(1) serial correlation is the Durbin Watson test. The forumla is

19. Introduction to regression analysis
Testing Hypotheses
Durbin Watson
We can refer to Savin, N.E., and K.J. White (1977), "The Durbin-Watson Test for Serial Correlation with Extreme Sample Sizes or manay Regressors," Econometrica 45, Or You can refer to to obtain the upper and lower of DW for 5% and 1% significance level.
For example, suppose we choose a 5% significance level with n = 45 and k = 4, d_u = 1.72 and d_l =
If DW < 1.336, we reject the null of no serial correlation (i.e. H_0) at 5% level.
If DW > 1.72, we fail to reject H_0.
If <= DW <= 1.72, the test is inconclusive.

20. Implement regression in MATLAB
Data: Hang Seng Bank Close Price, HSBC Close Price
Conduct a regression of HSBC close price (y) on Hang Seng Bank close price (x).
Suppose y = b_0 + b_1 x + u
where u is the residual
Matlab program: regression_tutorial.m

21. Implement regression in MATLAB
Consider the same model. Please use Regstats to find Coefficient, Standard Error, t-statistic, P-value of the t-stat, R-squared, Adjusted R-squared, Standard error of the regression, Sum of squared residuals, Durbin-Watson statistic, F-statistic, P-value of F-statistic
Matlab program: regression_tutorial2.m