BIOL 582 Supplemental Material Matrices, Matrix calculations, GLM using matrix algebra

Compact method of expressing mathematical operations (including statistics) Generalize from one to many variables (i.e. vectors to matrices) Matrix operations have geometric interpretations in data spaces Many data types (e.g., morphometric data) cannot be measured with a single variable, so multivariate methods are required to properly address hypotheses BIOL 582 Why bother with matrices?

Scalar: a number Vector: an ordered list (array) of scalars (n rows x 1 cols ) Matrix: a rectangular array of scalars (n rows x p cols ) BIOL 582 Scalars, vectors, and matrices

Reverse rows and columns Represent by A t or A′ Vector transpose works identically BIOL 582 Matrix transpose

Matrices must have same dimensions Add/subtract element-wise Vector addition/subtraction works identically Addition Subtraction BIOL 582 Matrix addition and subtraction

inner Scalar multiplication: Multiply scalar by each element in matrix or vector Matrix/vector multiplication is a summed multiplication Inner dimensions allow multiplication Outer dimensions determine size of result Order of matrices makes a difference: AB  BA BIOL 582 Matrix multiplication AB n 1 × p 1 * n 2 × p 2

Scalar multiplication: Matrix multiplication: Inner dimensions MUST AGREE!!! BIOL 582 Matrix multiplication

Inner (scalar) product: vector multiplication resulting in a scalar (weighted linear combination) Outer (matrix) product: vector multiplication resulting in a matrix Inner Product Outer Product Inner dimensions MUST AGREE!!! BIOL 582 Matrix multiplication: inner and outer products

I: Identity matrix (equivalent to ‘ 1 ’ for matrices) 1: A matrix of all ones 0: A matrix of all zeros Diagonal: diagonal contains non-zero elements Square: n = p Symmetric: off-diagonal elements same: BIOL 582 Special matrices

BIOL 582 Special matrices

Orthogonal: square matrix with property: VERY useful for statistics and other fields (e.g, morphometrics) Orthogonal matrices can be thought of as rigid rotations of data sets (shown later) Orthonormal Example: BIOL 582 Special matrices

Can ’ t divide matrices, so calculate the inverse (reciprocal) of denominator and multiply Inverses have property that: Inverses are tedious to calculate, so in practice we use a computer Only works for square matrices whose determinant  0!!! Determinant: combination of diagonal and off-diagonal elements A matrix whose determinant = 0 is Singular (has no inverse) For: BIOL 582 Matrix inversion

For the 2 x 2 case: Example: Confirm: BIOL 582 Matrix inversion: example

Multiplying data and other matrices has geometric interpretations XI=X: No change to X cIX=Y: Change of scale (e.g, enlargement) XD=Y: Stretching if D is diagonal XT=Y: Rigid rotation if T is p×p orthogonal XT=Y: Shear if T is not orthogonal (T can be decomposed into rotation, dilation, rotation) X = data matrix BIOL 582 Matrix multiplication: geometric interpretations

(images from C.A.B. Smith, 1969) Original Scalar (2) Scalar (1/2) Rotations Shears and Projections BIOL 582 Matrix multiplication: visual examples

The GLM model: Independent Variable/s: Dependent Variable/s: Solve for ‘regression coefficients’  found from: BIOL 582 GLM in matrix form Note, in general vectors are lower case and matrices are upper case, but using upper case is more encompassing

Multiply by inverse: Why this equation? Start with: Make ‘ X ’ a square matrix: BIOL 582 GLM in matrix form: Solving for β This is the model

and: 1. Expand matrixes: where: 2. Begin rewrite: BIOL 582 GLM in matrix form: Deriving univariate regression

2. From before: 4. Multiply 3. Calculate inverse: BIOL 582 GLM in matrix form: Deriving univariate regression

F-ratio is: SSM/SSE (with df corrections) Need to calculate full and reduced model SS Full model (contains all terms) Reduced model (X # has 1 less term – column of x values – in it) Significance based on: Or one can always use a random permutation approach… BIOL 582 GLM in matrix form: Calculating sums of squares (SS)

The Data: (for matrix form): SourcedfSSMS (SS/df)FP RegressionΔk = 1SSEr -SSEf = < Errorn - Δk – 1 = 3SSEf = Totaln – 1 = 4SST = SSEr = 10 BIOL 582 Regression example

BIOL 582 Using GLM for ANOVA Analysis of Variance (ANOVA) is the standard way of comparing means among multiple groups. ANOVA is the cornerstone of most applied stats courses in life science fields Linear regression equation ANOVA equation

Same idea, but must use special X-matrix coding Recode k groups in k-1 dummy variables columns) of X Generally, column 1 yields, column 2 yields deviation from for mean of group 1, etc. BIOL 582 Using GLM for ANOVA

The Data: n 1 =5 n 2 =5 BIOL 582 Using GLM for ANOVA: Example 1 SourcedfSSMS (SS/df)FP GroupΔk = 1SSEr -SSEf = Errorn - Δk – 1 = 8SSEf = 45 Totaln – 1 = 9SST = SSEr = 14 (Reduced model is one column of 1s) DEMO

BIOL 582 GLM final comments As we will learn, ANOVA, ANCOVA, Multiple Regression, MANOVA, MANCOVA, and Multivariate Multiple Regression, are all variants of the same GLM procedure. All of these “ different ” analytical approaches are no different to a computer using matrix calculations to perform GLM If there are 4 groups, then 4 – 1 = 3 dummy variables are needed. If there are 88 groups, then 88 – 1 = 87 dummy variables are needed. ALWAYS, there are a -1 “ factor ” levels for a groups.

BIOL 582 GLM final comments Dummy variables are “ indicator ” variables E.g., can be written as where Z is an indicator: 1 if in the group; 0 if not in the group. There are two ways to form the design matrix (X): Groups 2-4 means are expressed as deviations from the first group mean All group means are expressed as deviations from the overall group mean Analytically, these are no different, but different software packages use different approaches!