Matrix Algebra and Regression a matrix is a rectangular array of elements m=#rows, n=#columns  m x n a single value is called a ‘scalar’ a single row is called a ‘row vector’ a single column is called a ‘column vector’ a 13 = 6 matrix element B =

Matrix Algebra and Regression a square matrix has equal numbers of rows and columns in a symmetric matrix, a ij = a ji in a diagonal matrix, all off-diagonal elements = 0 an identity matrix is a diagonal matrix with diagonals = 1 I =

Trace The trace of a matrix is the sum of the elements on the main diagonal A = tr(A) = = 20

Matrix Addition and Subtraction The dimensions of the matrices must be the same

Matrix Multiplication X = A m x n B n x p C m x p C 11 = 2*2 + 5*5 + 1*1 + 8*8 = 94 The number of columns in A must equal the number of rows in B The resulting matrix C has the number of rows in A and the number of columns in B Note that the commutative rule of multiplication does not apply to matrices: A x B ≠ B x A

Transpose a Matrix Multiplying A x A′ above will give the uncorrected sums of squares for each row in A on the diagonal of a 2 x 2 matrix, with the sums of crossproducts on the off-diagonals

Invert a Matrix The inverse of a matrix is analogous to division in math An inverted matrix multiplied by the original matrix will give the identity matrix M -1 M = M -1 M =I It is easy to invert a diagonal matrix:

Calculate the Determinant (D) of the matrix M Verify The extension to larger matrices is not simple – use a computer! Inverting a 2x2 Matrix M = |M| = D = ad - bc M -1 = M = D = 2*9 – 5*3 M -1 =

Linear Dependence M = D = ad - bc M = D = 2*9 – 6*3 = 0 The matrix M on the right is singular because one row (or column) can be obtained by multiplying another by a constant. A singular matrix will have D=0. The rank of a matrix = the number of linearly independent rows or columns (1 in this case). A nonsingular matrix is full rank and has a unique inverse. A generalized inverse (M – ) can be obtained for any matrix, but the solution will not be unique if the matrix is singular. MM – M = M

Regression in Matrix Notation Y = X  + ε b = (X’X) -1 X’Y Linear model Parameter estimates SourcedfSSMS Regressionpb’X’YMS R Residualn-pY’Y - b’X’YMS E TotalnY’Y