Trees Example More than one variable

The residual plot suggests that the linear model is satisfactory. The R squared value seems quite low though, so from physical arguments we force the line to pass through the origin.

The R squared value is higher now, but the residual plot is not so random.

We might now ask if we can find a model with both explanatory variables height and girth. Physical considerations suggest that we should explore the very simple model Volume = b 1 × height × (girth) 2 +  This is basically the formula for the volume of a cylinder.

So the equation is: Volume = × height × (girth) 2 + 

The residuals are considerably smaller than those from any of the previous models considered. Further graphical analysis fails to reveal any further obvious dependence on either of the explanatory variable girth or height. Further analysis also shows that inclusion of a constant term in the model does not significantly improve the fit. Model 4 is thus the most satisfactory of those models considered for the data.

However, this is regression “through the origin” so it may be more satisfactory to rewrite Model 4 as volume = b 1 +  height × (girth) 2

so that b 1 can then just be regarded as the mean of the observations of volume height × (girth) 2 recall that  is assumed to have location measure (here mean) 0.

Compare with found earlier

Practical Question 2 yx1x1 x2x

So y = x x2 + e

Use >plot(multregress) or >plot(cooks.distance(multregress),type="h")

> ynew=c(y,12) > x1new=c(x1,20) > x2new=c(x2,100) > multregressnew=lm(ynew~x1new+x2new)

Very large influence

Second Example > ynew=c(y,40) > x1new=c(x1,10) > x2new=c(x2,50) > multregressnew=lm(ynew~x1new+x2new)

Multiple Linear Regression - Matrix Formulation Let x = (x 1, x 2, …, x n )′ be a n  1 column vector and let g(x) be a scalar function of x. Then, by definition,

For example, let Let a = (a 1, a 2, …, a n )′ be a n  1 column vector of constants. It is easy to verify that and that, for symmetrical A (n  n)

Theory of Multiple Regression Suppose we have response variables Y i, i = 1, 2, …, n and k explanatory variables/predictors X 1, X 2, …, X k. i = 1,2, …, n There are k+2 parameters b 0, b 1, b 2, …, b k­ and σ 2

X is called the design matrix

OLS (ordinary least-squares) estimation

Fitted values are given by H is called the “hat matrix” (… it puts the hats on the Y’s)

The error sum of squares, SS RES, is The estimate of  2 is based on this.

Example: Find a model of the form yx1x1 x2x for the data below.

X is called the design matrix

The model in matrix form is given by: We have already seen that Now calculate this for our example

R can be used to calculate X’X and the answer is:

To input the matrix in R use X=matrix(c(1,1,1,1,1,1,1,3.1,3.4,3.0,3.4, 3.9,2.8,2.2,30,25,20,30,40,25,30),7,3) Number of rows Number of columns

Notice command for matrix multiplication

The inverse of X’X can also be obtained by using R

We also need to calculate X’Y Now

Notice that this is the same result as obtained previously using the lm result on R

So y = x x2 + e

The “hat matrix” is given by

The fitted Y values are obtained by

Recall once more we are looking at the model

Compare with

Error Terms and Inference A useful result is : n : number of points k: number of explanatory variables

In addition we can show that: And c (i+1)(i+1) is the (i+1)th diagonal element of where s.e.(b i )=  c (i+1)(i+1) 

For our example:

was calculated as:

This means that c 11 = 6.683, c 22 =0.7600,c 33 = Note that c 11 is associated with b 0, c 22 with b 1 and c 33 with b 2 We will calculate the standard error for b 1 This is  x =

The value of b 1 is Now carry out a hypothesis test. H 0 : b 1 = 0 H 1 : b 1 ≠ 0 The standard error of b 1 is ^

The test statistic is This calculates as ( – 0)/ = 3.55

Ds….. ………. t tables using 4 degrees of freedom give cut of point of for 2.5%. ………………

We therefore accept H 1. There is no evidence at the 5% level that b 1 is zero. The process can be repeated for the other b values and confidence intervals calculated in the usual way. CI for  2 - based on the  4 2 distribution of ((4  )/11.14, (4  )/0.4844) i.e. (0.030, 0.695)

The sum of squares of the residuals can also be calculated.