Correlation

Correlation Coefficient Array

Correlation Multiple regression Polynomial regression Multivariate transformations

Multiple Regression

Regression Until now we have been concerned with the relationship between two variables Where more complex relationships are concerned it is often better to consider multiple regression or more correctly, multiple linear regression.

Where e is a random error term Theory Consider simplest form of multiple regression where y is the dependant variable and x1 and x2 independent variables y = b0 + b1x1 + b2x2 + e Where e is a random error term

Theory y = b0 + b11x11 + b21x21 + e1

Theory y1 = b0 + b11x11 + b21x21 + e1 y2= b0 + b12x12 + b22x22 + e2 : : : : : : yi = b0 + b1ix1i + b2ix2i + ei yn = b0 + b1nx1n + b21x2n + en yi = nb0 + b1x1i + b2x2i

Theory yi = nb0 + b1x1i + b2x2i Least Square “Best” Fit Minimize the sum of squares of error F = ei2 = [yi - b0 - b1x1i - b2x2i]2

F = ei2 = [yi - b0 - b1x1i - b2x2i]2 Theory F = ei2 = [yi - b0 - b1x1i - b2x2i]2 dF/db1 = x1y - b2x1x2 - b1x12 and dF/db2 = x2y - b1x1x2 - b2x22

b1 = [(x22)(x1y)-(x1x2)(x2y)] Theory dF/db1 = x1y - b2x1x2 - b1x12 dF/db2 = x2y - b1x1x2 - b2x22 b1 = [(x22)(x1y)-(x1x2)(x2y)] [(x12)x22)-x1x2)2]

Theory dF/db1 = x1y - b2x1x2 - b1x12 b1 = [(x22)(x1y)-(x1x2)(x2y)] [(x12)(x22)-(x1x2)2] b2 = [(x12)(x2y)-(x1x2)(x1y)] [(x22)(x12)-(x1x2)2]

b0 = mean(y)-b1mean(x1)-b2mean(x2) Theory b0 = mean(y)-b1mean(x1)-b2mean(x2)

Analysis of Variance Total SS = y2 - [y]2/n Reg SS = b1x1y + b2x2y More generally : Reg SS = [bixiy] Residual SS = Total SS - Reg SS

Analysis of Variance Table

Example

Example x12=1,753.7; x22=23.2; x1y=-65.2  x2y=7,210.0; x1x2=-156.7; y2=3,211,504

b1 = [(x22)(x1y)-(x1x2)(x2y)] Example x12=1,753.7; x22=23.2; x1y=-65.2  x2y=7,210.0; x1x2=-156.7; y2=3,211,504 b1 = [(x22)(x1y)-(x1x2)(x2y)] [(x12)(x22)-(x1x2)2]

b1 = [(x22)(x1y)-(x1x2)(x2y)] Example x12=1,753.7; x22=23.2; x1y=-65.2  x2y=7,210.0; x1x2=-156.7; y2=3,211,504 b1 = [(x22)(x1y)-(x1x2)(x2y)] [(x12)x22)-x1x2)2] b1 = [(23.2)(-65.2)-(-156.7)(7210)] [(1753.7)(23.2)-(-156.6)2] = -23.75

b2 = [(x12)(x2y)-(x1x2)(x1y)] Example x12=1,753.7; x22=23.2; x1y=-65.2  x2y=7,210.0; x1x2=-156.7; y2=3,211,504 b2 = [(x12)(x2y)-(x1x2)(x1y)] [(x12)(x22)-(x1x2)2] b2 = [(1753.7)(-7210)-(-156.7)(-65,194)] [(1753.7)(23.2)-(-156.6)2] = -150.27

Example b0 = 6561 - (-23.75)(96.2) - (150.27)(16.7) = 6336 y = 6336 - 23.75 x1 + 150.27 x2

Analysis of Variance Table

Analysis of Variance Table

Analysis of Variance Table

Analysis of Variance Table

Analysis of Variance Table

Analysis of Variance Table

Matrix Formation of Multiple Regression y = b0 + b1x1 + b2x2 + ….. + bnxn e Y = X x b + E

Introduction to Matrixes } 6b1 + 3b2 = 24 4b1 + 4 b2 = 20 Simultaneous equations [ ] [ ] [ ] } 6 3 b1 24 4 4 b2 20 Matrix Form x =

Matrix Formation y = b0 + b1x1 + b2x2 + ….. + bnxn e Y = X x b + e

Matrix Formation Y = X x b

F = ee’ = YY’ - 2YX’b + bb’ XX’ Matrix Formation F = ee’ = YY’ - 2YX’b + bb’ XX’ dF/db = 2XX’ b - 2YX’ = 0 XX’b = YX’

Two Variable Example = x11 x21 x31 x41 x51 x11 x12 x12 x1x2

Matrix Formation XX’ =

Two Variable Example x11 x12 x21 x22 y1 y2 x1y x2y x =

Matrix Formation = YX’

[ ] [ ] [ ] Two Variable Example x12 x1x2 b1 x1y x2x1 x22 b2 x2y = XX’ x b = YX’ (XX’)-1XX’ x b = (XX’)-1YX’ b = (XX’)-1YX’

Matrix Formation b = (XX’)-1 YX’ Find the inverse of XX’ Donated by (XX’)-1 b = (XX’)-1 YX’

Matrix Inverse with Two Variables A x A-1 = [U]

Matrix Inverse with Two Variables A x A-1 = [U] [ ] [ ] [ ] a b c d 1 ad-bc d -b -c a 1 0 0 1 = x

Matrix Inverse with Two Variables [ ] [ ] [ ] x12 x1x2 b1 x1y x2x1 x22 b2 x2y x =

Matrix Inverse with Two Variables [ ] [ ] [ ] x12 x1x2 b1 x1y x2x1 x22 b2 x2y x = XX’ x b = X’Y

Matrix Inverse with Two Variables [ ] [ ] [ ] x12 x1x2 b1 x1y x2x1 x22 b2 x2y x = XX’ x b = X’Y [ ] [ ] x12 x1x2 x22 - x1x2 x2x1 x22 - x2x1 x12 1 ad-bc = [U] x XX’ x (XX’)-1 = Unit

Matrix Inverse with Two Variables [ ] [ ] [ ] x22 - x1x2 x1y b1 -x2x1 x12 x2y b2 1 ad-bc = x (XX’)-1 x Y = b = x22 x12 - [x2x1]2 1 ad-bc

Compare Matrix with None b1 = [(x22)(x1y)-(x1x2)(x2y)] [(x12)(x22)-(x1x2)2] b2 = [(x12)(x2y)-(x1x2)(x1y)] [(x22)(x12)-(x1x2)2]

Forward Step-Wise Regression

Two Variable Multiple Regression

Analysis of Variance Table y = 6336 - 23.75 x1 + 150.27 x2

Two Variable Multiple Regression There is significant regression effects by regressing both independent variables onto the dependant variable. The is significant linear relationship between height (x1) and yield but no relationship between yield and tiller. There is significant linear relationship between tiller (x2) and yield and no relationship between yield and height.

Two Variable Multiple Regression We may have made the relationship too complex by including both variables. Forward Step-wise Regression. Backward Step-wise Regression.

Two Variable Multiple Regression

Analysis of Variance Table y = 10,131 - 37.11 Height (x1)

Analysis of Variance Table y = 6336 - 23.75 Height + 150.27 Tiller

Analysis of Variance Table y = 6336 - 23.75 Height + 150.27 Tiller

Analysis of Variance Table y = 6336 - 23.75 Height + 150.27 Tiller

Analysis of Variance Table y = 10,131 - 37.11 Height (x1)

Forward Step-Wise Regression Example 2 20 Spring Canola Cultivars Average over 10 environments Seed yield; plant establishment; days to first flowering, days to end of flowering; plant height; and oil content

Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 1.00 F.Start -0.30 F.Finish -0.15 0.93 Height -0.45 0.72 0.70 0.04 -0.51 -0.52 -0.27 0.31 -0.82 -0.80 -0.53 -0.21

Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 1.00 -0.30 -0.15 -0.45 0.04 0.31 F.Start 0.93 0.72 -0.51 -0.82 F.Finish 0.70 0.52 0.80 Height -0.27 -0.53 -0.52 -0.21 -0.80

Analysis of Variance Table y = 3,194 - 32.9 x F.Start

A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 1.00 -0.30 -0.15 -0.45 0.04 0.31 F.Start 0.93 0.72 -0.51 -0.82 F.Finish 0.70 0.52 0.80 Height -0.27 -0.53 -0.52 -0.21 -0.80 A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x

A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 1.00 -0.30 -0.15 -0.45 0.04 0.31 F.Start 0.93 0.72 -0.51 -0.82 F.Finish 0.70 0.52 0.80 Height -0.27 -0.53 -0.52 -0.21 -0.80 A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x

A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 1.00 -0.30 -0.15 -0.45 0.04 0.31 F.Start 0.93 0.72 -0.51 -0.82 F.Finish 0.70 0.52 0.80 Height -0.27 -0.53 -0.52 -0.21 A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x

A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 1.00 -0.30 -0.15 -0.45 0.04 0.31 F.Start 0.93 0.72 -0.51 -0.82 F.Finish 0.70 0.52 0.80 Height -0.27 -0.53 -0.52 -0.21 A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x

A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 1.00 -0.30 -0.15 -0.45 0.04 0.31 F.Start 0.93 0.72 -0.51 -0.82 F.Finish 0.70 0.52 0.80 Height -0.27 -0.53 -0.52 -0.21 -0.80 -0.63 A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x

A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 0.91 0.38 -0.23 0.11 0.06 F.Start F.Finish 0.36 0.03 -0.05 0.04 Height 0.48 0.10 0.74 0.21 -0.63 0.33 A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x

Analysis of Variance Table y = 5335 - 38.4 F.Start - 57.7 %Oil

A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x Example #2 Character Est. F.St. F.Fi. Ht. %Oil Yield Establish 0.90 0.39 -0.23 0.04 F.Start F.Finish 0.36 Height 0.48 0.11 0.05 0.29 A[i,j] = A[i,j]–{Ai,x x Ax,j}/Ax,x

Analysis of Variance Table y = 6779 - 30.4 FS - 63.0 %Oil + 8.7 Height

Analysis of Variance Table y = 5335 - 38.4 F.Start - 57.7 %Oil

Forward Step-Wise Regression Enter the variable “most associated with the dependant variable. Check to see if relationship is significant. Adjust the relationship between the dependant variable and the other remaining variables, accounting for the relationship between the dependant variable and the entered variable(s).

Forward Step-Wise Regression

Forward Step-Wise Regression Enter most correlated variable

Forward Step-Wise Regression Enter most correlated variable Check that entry is significant

Forward Step-Wise Regression Enter most correlated variable Check that entry is significant Adjust correlation with other variables

Forward Step-Wise Regression Enter most correlated variable Check that entry is significant Adjust correlation with other variables

Polynomial Regression

Polynomial Regression

Polynomial Regression

Polynomial Regression

Analysis of Variance Table y = -36.25 + 15.730 N - 0.218 N2

Polynomial Regression y = -36.25 + 15.730 N - 0.218 N2 dY/dN = Slope

Polynomial Regression y = -36.25 + 15.730 N - 0.218 N2 dy/dN = +15.730 - 0.436 N 0.436 N = 15.730 n = 36.08

Multivariate Transformation