1. Review of Matrix Algebra
Department of Statistics
Texas A&M University
presented by Curtis Alexander

2. What is a Matrix ?
A matrix is a rectangular table, or array, composed of either
numbers OR
variables
A matrix may also contain
fractions
AND/OR
decimals

3. Matrix Notation I Matrices (the plural of matrix) are enclosed in
brackets (as we have seen) OR
parenthesis (less commonly used notation)
Note: We will use the more conventional bracket notation.

4. Matrix Notation II A matrix is typically denoted by a capital letter.
When a matrix appears inline with text, it is often written in bold.
Note: Different textbooks will use different notation for matrices – so check the notation!
A =
In simple linear regression, we call H the hat matrix because it transforms the vector of observed responses into the vector of fitted responses.
bold

5. Elements of a Matrix : NO!!
The individual numbers (or variables) in a matrix are called elements.
Each possible location in a matrix must contain an element.
: Is this a matrix ?
: NO!!
element
missing element
missing element

6. Rows and Columns : What other parts of a matrix are named?
Rows – run horizontally
AND
Columns – run vertically
Row 2
Row 4

7. Location of Elements t23 T =
We refer to a specific element in a matrix by referencing its location using rows and columns.
When referencing an element within a matrix, we use the lower case letter of the matrix.
We list the row first, followed by the column.
Thus we say that the element t23 = 5 .
T =

8. Elements of a Matrix Practice
: w12 = ?
: w12 = 5
: 6 = ?
: 6 = w24
: The element with value 16 is located in which row?
: Row 3
: How many columns (total) does this matrix contain?
: 4 columns
W =

9. Size of a Matrix
A =
The size of a matrix is expressed in the form r rows by p columns.
Matrix A has 4 rows and 3 columns, so we list its size as 4 x 3 – which we read “four by three” and which we write as A4x3.
Note: Dimension is another word used for size of a matrix.
: What is the size (or dimensions) of matrix X?
: 2 x 2
X =

10. Special Matrices I
Vector – a matrix that has only one row or one column
Column vector – matrix with one column
Row vector – matrix with one row
Note: Vectors are denoted by lower case letters.
Scalar – a matrix that has only one row and one column, or alternatively a matrix that only has one element
Scalars are usually written without brackets.
Think of scalars as merely constants.
d =
column vector
row vector
g = β
scalars 11

11. Special Matrices II diagonal
Square matrix – the number of rows and columns are equal
Diagonal – the elements that run from the upper left element to the lower right element in a square matrix – also called the main diagonal
Diagonal matrix – a square matrix with all non-diagonal elements equal to zero
diagonal
diagonal matrix

12. Special Matrices III I =
Zero matrix – a matrix with all elements equal to zero
Identity matrix – a diagonal matrix with all elements along the diagonal equal to one
The identity matrix is denoted by the capital letter I.
The identity matrix will become important later when we discuss the inverse of a matrix.
zero matrix
I =
identity matrix

13. Matrix Addition
A =
Matrices may only be added if they are the same size.
A and B are both 2x3 so they may be added.
The result of A+B, which we call C, is also size 2x3.
Note: Matrix addition IS commutative in general – A+B = B+A
Take each matching element of A and B and add them together, placing the sum in the same elemental position of C.
B =
C = =

14. Matrix Subtraction D = E = F = =
Like matrix addition, matrices may only be added if they are the same size.
D and E are both 2x3 so they may be subtracted.
The result of D - E, which we call F, is also size 2x3.
Note: Matrix subtraction is NOT commutative in general – D - E ≠ E - D
Take the matching element of D and E and subtract the element from matrix E from the element from matrix D, placing the difference in the same elemental position of F.
E =
F = =

15. Matrix Add/Sub Practice
: A – B = ? :
: C + A = ?
: B + D = ?
: Sorry – trick question! We cannot sum because their sizes are different!
A =
B =
C =
D =

16. Matrix Multiplication Introduction
To multiply matrices, the # of columns of the first matrix MUST equal the # of rows of the second matrix.
How do we easily go about determining if M can be multiplied by N?
Write the dimensions of M adjacent to the dimensions of N. If the inner dimensions match, then they may be multiplied.
The outer dimensions determine the dimensions of the product P.
Note: Matrix multiplication is NOT commutative in general --
M*N ≠ N*M
N =
outer dimensions M N
2 x 2
2 x 3
inner dimensions
P =

17. Matrix Multiplication Properties
If A, B, and C are matrices and if the multiplicative combinations below are assumed to have the correct size then in general
(AB)C = A(BC) associativity
(A+B)C = AC + BC left distributivity
C(A+B) = CA + CB right distributivity
However, commutativity is does NOT hold in the general case
AB ≠ BA

18. Multiplication – Scalar * Matrix
This is the easiest form of matrix multiplication – a scalar, or constant, multiplied by a matrix.
Simply multiply each element in the matrix by the scalar. = =

19. Multiplication – Vector * Vector
The only vectors that can be multiplied together are
row vector * column vector
column vector * row vector
Check to ensure that the # columns (or rows) of the vector equals the # rows (or columns) of the vector.
To evaluate j*k, multiply the first element of j by the first element of k. The result is placed in l11. Next multiply the first element of j by the second element of k and place it in l12.
This is continued where the element location of j becomes the row location of the result and the element location of k becomes the column location of the result. j k
2 x 1
1 x 2
equal
column vector
j =
row vector
k =
l =

20. Multiplication – Vector * Matrix IB
1 x 3
3 x 3
equal
Only row vectors can be multiplied by a matrix. Check to ensure that the # columns of the vector equals the # rows of the matrix.
To evaluate a*B = c, first multiply the elements of vector a that correspond with the elements of the first column of B. Second, sum all three of these intermediate product results to get the final result. This result becomes the first element of c (c11).
The result of any vector multiplied by a matrix is a vector.
B =
a =
c = =

21. Multiplication – Vector * Matrix II
To further evaluate a*B = c, multiply the elements of vector a that correspond with the elements of the second column of B. Sum all three of these intermediate product results to get the final result. This result becomes the second element of c (c12).
a =
B =
c = =

22. Multiplication – Vector * Matrix III
To finish evaluating a*B = c, multiply the elements of vector a that correspond with the elements of the third column of B. Sum all three of these intermediate product results to get the final result. This result becomes the third element of c (c13).
a =
B =
c = = =

23. Multiplication – Matrix * Matrix I
To multiply matrices, the # columns of the first matrix must equal the # rows of the second matrix.
We proceed similar as to when multiplying a vector*matrix in order to multiply X*Y = Z.
First, multiply the first row of matrix X by the first column of matrix Y (matching the elements as we have done previously). Sum these products and the result goes in z11. X Y
2 x 2
2 x 3
equal
Y =
X =
2*5+3*1 = 13
Z =

24. Multiplication – Matrix * Matrix II
To find z21, multiply the second row of matrix X by the first column of matrix Y (matching the elements as we have done previously). Sum these products and the result goes in z21.
Note: For a specific element in Z, say z21, we know this means multiply the second row of matrix X by the first column of matrix Y.
Y =
X =
Z =
4*5+5*1 = 25
z21
1st column of Y
2nd row of X

25. Multiplication – Matrix * Matrix III
For all subsequent elements of Z, multiply the appropriate row of matrix X by the appropriate column of matrix Y (matching the elements as we have done previously). Sum these products and the result is placed in the appropriate location in Z.
As a final example, to find z13 multiply the first row of matrix X by the third row of matrix Y and then sum the products of these rows and columns.
X =
Y =
2*2+3*3 = 13
Z =

26. Matrix Multiplication Practice
: b*A = ? :
: C*E = ?
: Which of the following products exist?
A*b; A*D; b*C; E*C; A*C; D*D; b*b
: A*D; b*C; A*C; D*D
A =
b =
C =
D =
E =

27. Trace
sum these elements
U =
To find the trace of a square matrix, simply sum all the elements that lie along the main diagonal of the matrix.
V =
sum these elements

28. Transpose A = To take the transpose of a matrix, either
write the rows as columns OR
write the columns as rows
The transpose of a column vector is a row vector and vice versa.
Note: Transpose may either be written with a capital T or using an apostrophe.
AT =
h =
h ́ =

29. Inverse Introduction I
We have seen addition, subtraction, and multiplication of matrices. There does not exist division of matrices per se – instead we shall use the inverse of a matrix.
For example, in order to solve for x at right, you simply divide both sides of the equation by three and find that x=4.
Another way we say this is accomplished is by multiplying both sides of the equation by 1/3.

30. Inverse Introduction II
If A is a square matrix and the inverse of A exists, then
A*A-1 = A-1*A = I (which is similar to
the algebraic expression ⅓ * 3 = 1).
We say that A is invertible if A*A-1= I.
Note: Not every square matrix has an inverse. If a matrix, say A, does not have an inverse then we say that the inverse of A does not exist.

31. Determinant of a 2x2 Matrix
Before we can get to calculating the inverse of a matrix, we need to know how to calculate the determinant.
To calculate the determinant, simply multiply the opposite corner elements and subtract the product results.
Y =

32. Inverse of a 2x2 Matrix I
A =
First, calculate the determinant of the matrix you would like to invert.
Next, invert the signs of element a12 and a21 and then exchange the values of a11 and a22.
Finally, multiply by this new matrix.
sign change
values exchanged
sign change
A-1 = =

33. division by 0 – therefore inverse does not exist
Inverse of a 2x2 Matrix II
X =
For a generic matrix X, the formula for calculating the inverse is at right.
When does the inverse not exist?
Whenever the determinant is 0 because we would be dividing by 0 in the inverse equation.
X-1 =
division by 0 – therefore inverse does not exist
Z =
Z-1 =

34. Determinant and Inverse Practice
: det(A)= ?
: 4*5 – 3*7=-1
: D-1=? :
: det(B)=?
: 1*2 – 3*2=-4
: C-1=?
: The inverse does not exist because the determinant is 0 --
det(C)=4*5-2*10=0
A =
B =
C =
D =

35. Writing a System of Linear Equations in Matrix Form
What if we had the following system of linear equations.
How could we write this in matrix notation?
First, write the coefficients in a 2x2 matrix (or an nxn matrix depending upon the number of equations).
Next, create a column vector containing the variables and place it to the right of the coefficient matrix.
Finally create another column vector containing the values on the right of the equal sign.

36. Solving Simple Matrix Equations
To solve equations like A*X=Y for X, we do not divide Y by A, but multiply the inverse of A (which is written as A-1) by Y to find X.
Y =
identity matrix
multiply on left
inverse of A

37. Example from Multiple Linear Regression I
Where do we see matrices in statistics?
One such place is in multiple linear regression, which is seen at right. This is a multiple linear regression model for n independent observations generated from p predictor variables.
Instead of writing out n equations, this may be written using matrices as

38. Example from Multiple Linear Regression II
As a final example of where matrices occur in statistics, we look again at multiple linear regression.
The residual sum of squares of β is seen at right.
Using RSS(β), we calculate the least squares estimate of β. In this form, we obtain the least squares estimate of each βi with just a few matrix computations.

39. Matrix Calculator Web based Matrix Calculator
Note that there are other matrix calculators online, but this is among the easiest to use.
Now I will demonstrate some of its capabilities.