1. Trees Example More than one variable

5. The residual plot suggests that the linear model is satisfactory. The R squared value seems quite low though, so from physical arguments we force the line to pass through the origin.

7. The R squared value is higher now, but the residual plot is not so random.

9. We might now ask if we can find a model with both explanatory variables height and girth. Physical considerations suggest that we should explore the very simple model Volume = b 1 × height × (girth) 2 +  This is basically the formula for the volume of a cylinder.

11. So the equation is: Volume = 0.002108 × height × (girth) 2 + 

14. The residuals are considerably smaller than those from any of the previous models considered. Further graphical analysis fails to reveal any further obvious dependence on either of the explanatory variable girth or height. Further analysis also shows that inclusion of a constant term in the model does not significantly improve the fit. Model 4 is thus the most satisfactory of those models considered for the data.

15. However, this is regression “through the origin” so it may be more satisfactory to rewrite Model 4 as volume = b 1 +  height × (girth) 2

16. so that b 1 can then just be regarded as the mean of the observations of volume height × (girth) 2 recall that  is assumed to have location measure (here mean) 0.

17. Compare with 0.002108 found earlier

18. Practical Question 2 yx1x1 x2x2 3.53.130 3.23.425 3.0 20 2.93.230 4.03.940 2.52.825 2.32.230

19. So y = -0.2138 + 0.8984x1 + 0.01745x2 + e

21. Use >plot(multregress) or >plot(cooks.distance(multregress),type="h")

22. > ynew=c(y,12) > x1new=c(x1,20) > x2new=c(x2,100) > multregressnew=lm(ynew~x1new+x2new)

25. Very large influence

26. Second Example > ynew=c(y,40) > x1new=c(x1,10) > x2new=c(x2,50) > multregressnew=lm(ynew~x1new+x2new)

30. Multiple Linear Regression - Matrix Formulation Let x = (x 1, x 2, …, x n )′ be a n  1 column vector and let g(x) be a scalar function of x. Then, by definition,

31. For example, let Let a = (a 1, a 2, …, a n )′ be a n  1 column vector of constants. It is easy to verify that and that, for symmetrical A (n  n)

32. Theory of Multiple Regression Suppose we have response variables Y i, i = 1, 2, …, n and k explanatory variables/predictors X 1, X 2, …, X k. i = 1,2, …, n There are k+2 parameters b 0, b 1, b 2, …, b k­ and σ 2

33. X is called the design matrix

35. OLS (ordinary least-squares) estimation

37. Fitted values are given by H is called the “hat matrix” (… it puts the hats on the Y’s)

38. The error sum of squares, SS RES, is The estimate of  2 is based on this.

39. Example: Find a model of the form yx1x1 x2x2 3.53.130 3.23.425 3.0 20 2.93.230 4.03.940 2.52.825 2.32.230 for the data below.

40. X is called the design matrix

41. The model in matrix form is given by: We have already seen that Now calculate this for our example

42. R can be used to calculate X’X and the answer is:

43. To input the matrix in R use X=matrix(c(1,1,1,1,1,1,1,3.1,3.4,3.0,3.4, 3.9,2.8,2.2,30,25,20,30,40,25,30),7,3) Number of rows Number of columns

46. Notice command for matrix multiplication

47. The inverse of X’X can also be obtained by using R

48. We also need to calculate X’Y Now

49. Notice that this is the same result as obtained previously using the lm result on R

50. So y = -0.2138 + 0.8984x1 + 0.01745x2 + e

51. The “hat matrix” is given by

52. The fitted Y values are obtained by

53. Recall once more we are looking at the model

54. Compare with

55. Error Terms and Inference A useful result is : n : number of points k: number of explanatory variables

56. In addition we can show that: And c (i+1)(i+1) is the (i+1)th diagonal element of where s.e.(b i )=  c (i+1)(i+1) 

57. For our example:

58. was calculated as:

59. This means that c 11 = 6.683, c 22 =0.7600,c 33 =0.0053 Note that c 11 is associated with b 0, c 22 with b 1 and c 33 with b 2 We will calculate the standard error for b 1 This is  0.7600 x 0.2902 = 0.2530

60. The value of b 1 is 0.8984 Now carry out a hypothesis test. H 0 : b 1 = 0 H 1 : b 1 ≠ 0 The standard error of b 1 is 0.2530 ^

61. The test statistic is This calculates as (0.8984 – 0)/0.2530 = 3.55

62. Ds….. ………. t tables using 4 degrees of freedom give cut of point of 2.776 for 2.5%. ………………................

63. We therefore accept H 1. There is no evidence at the 5% level that b 1 is zero. The process can be repeated for the other b values and confidence intervals calculated in the usual way. CI for  2 - based on the  4 2 distribution of ((4  0.08422)/11.14, (4  0.08422)/0.4844) i.e. (0.030, 0.695)

64. The sum of squares of the residuals can also be calculated.