1. Linear Algebra: Matrices and Vectors – Part 3
By Dr. Samer Awad
Assistant professor of biomedical engineering
The Hashemite University, Zarqa, Jordan
Last update: 15 November 2018

2. 7.4 Linear Independence of Vectors2
November November 2018
7.4 Linear Independence of Vectors
• For a set of m vectors a(1), …, a(m) (with the same number of components), a linear combination of them is:
where c1, c2, …, cm are any scalars. Now consider the equation:
• Clearly, this equation holds if we choose all c’s zero, because then it becomes 𝟎=𝟎.

3. Linear Independence of Vectors3
November November 2018
Linear Independence of Vectors
• If this equation also holds with scalars not all zero, we call these vectors linearly dependent. Otherwise, they are linearly independent.
• Linear dependence means that we can express at least one of the vectors as a linear combination of the other vectors. For example, if c1≠0:

4. Example1: Linear Independence of Vectors4
November November 2018
Example1: Linear Independence of Vectors
• Although this is easily checked by vector arithmetic, it is not so easy to discover. However, a systematic method for finding out about linear independence and dependence will be explained below.

5. 5
November November 2018
Rank of a Matrix
• Definition: The rank of a matrix A is the maximum number of linearly independent row vectors of A. It is denoted by rank A.
• Definition: We call a matrix A1 row-equivalent to a matrix A2 if A1 can be obtained from A2 by finitely many elementary row operations
 Matrices in Gauss elim. are row-equiv
• Theorem 1: Row-Equivalent Matrices: Row-equivalent matrices have the same rank.

6. Example1: Rank of a Matrix6
November November 2018
Example1: Rank of a Matrix
• The following three vectors (given previously):
can be represented by the matrix:

7. Example1: Rank of a Matrix7
November November 2018
Example1: Rank of a Matrix
• The last matrix is in row-echelon form and has two nonzero rows. Hence rank A=2

8. Previous Examples: Rank of a Matrix8
November November 2018
Previous Examples: Rank of a Matrix
Hence, rank = 2
Hence, rank = 3
Assume everything here are raw vectors

9. Linear Independence of Vectors9
November November 2018
Linear Independence of Vectors
• Theorem 2: Linear Independence and Dependence of Vectors: p number of vectors are linearly independent if the matrix formed with these vectors has rank p. Otherwise linearly dependent.
Rank(A) = # of vectors  vectors are independent
• Theorem 3: Rank in Terms of Column Vectors: The rank r of a matrix A equals the maximum number of linearly independent column vectors of A.
• Hence A and its transpose AT have the same rank. Proof: see textbook.

10. Example1: Rank in Terms of Column Vectors10
November November 2018
Example1: Rank in Terms of Column Vectors
Recall the matrix from example1 above:
Performing the following column operations concludes -as before - that rank = 2:

11. Linear Independence of Vectors11
November November 2018
Linear Independence of Vectors
• Theorem 4: Linear Dependence of Vectors: Consider p vectors each having n components. If n<p then these vectors are linearly dependent.
• Eg:

12. Linear Independence of Vectors12
November November 2018
Linear Independence of Vectors
rref : Reduced row echelon form
>> rref ( [ ; ; ] )
ans =
>> rref ( [ ; ; ] )
>> rref ( [ ; ; ] )

13. 7.5 Solutions of Linear Systems:13
November November 2018
7.5 Solutions of Linear Systems:
Existence, Uniqueness
• Rank gives complete information about existence, uniqueness, and general structure of the solution set of linear systems as follows.
• A linear system of equations in n unknowns has:
– a unique solution if rank(𝑨) = rank( 𝑨 ) = n.
– infinitely many solutions if if rank(𝑨) = rank( 𝑨 ) < n.
– no solution if rank(𝑨) ≠ rank( 𝑨 ).
• If solutions exist, they can all be obtained by the Gauss elimination. (This method will automatically reveal whether or not solutions exist

14. Previous Examples: Rank of a Matrix14
November November 2018
Previous Examples: Rank of a Matrix
rank(𝑨) = rank( 𝑨 ) = n = 2  unique soln
rank(𝑨) = rank( 𝑨 ) = n = 3  unique soln
rank(𝑨) = rank( 𝑨 ) = 2 < (n=4)  infinite soln’s
rank(𝑨) ≠ rank( 𝑨 )  no soln

15. 15
November November 2018
Determinants
• Determinants were originally introduced for solving linear systems. They have important engineering applications in eigenvalue problems (Sec. 8.1), differential equations, vector algebra (Sec. 9.3), and in other areas.
• A determinant of order n is a scalar associated with an n × n matrix:

16. Second-Order Determinants16
November November 2018
Second-Order Determinants
• A determinant of second order can be defined by:

17. Third-Order Determinants17
November November 2018
Third-Order Determinants
• A determinant of third order can be defined by:

18. Third-Order Determinants18
November November 2018
Third-Order Determinants
• A determinant of third order can be defined by:
C11 : cofactor of a11

19. Third-Order Determinants19
November November 2018
Third-Order Determinants

20. n-order Determinants • A determinant is defined as follows:20
November November 2018
n-order Determinants
• A determinant is defined as follows:
“Column-wise” expansion
“Row-wise” expansion (shown in previous slide)

21. Minors and Cofactors • Cjk is the cofactor of ajk in D21
November November 2018
Minors and Cofactors
• Cjk is the cofactor of ajk in D
• Mjk is the minor of ajk in D (determinant of order n-1): the determinant of the submatrix of 𝑨 obtained from 𝑨 by omitting the row and column of the entry.
• Equation (3) above can be rewritten as :

22. Minors and Cofactors of Third-Order Determinants22
November November 2018
Minors and Cofactors of Third-Order Determinants
Checkerboard pattern for Cjk

23. Cofactor Matrix 𝑐𝑜𝑓(𝐴) = −7 −13 8 2 −2 2 3 7 −2 23
November November 2018
Cofactor Matrix
𝑐𝑜𝑓(𝐴) = −7 − − −2

24. 24
November November 2018
Cofactor Matrix

25. Adjoint (or Adjugate) Matrix25
November November 2018
Adjoint (or Adjugate) Matrix
• (𝑎𝑑𝑗 𝑨) = (𝑐𝑜𝑓 𝑨)𝑇
• 𝑨 (𝑎𝑑𝑗 𝑨) = (𝑎𝑑𝑗 𝑨) 𝑨 = (det⁡𝑨) 𝑰
• Using MATLAB:
Cofactor: cof(A)
Transpose: transpose(A) or A’
Adjoint: transpose(cof(A)) or cof(A)’

26. Example: Determinant of a Triangular Matrix26
November November 2018
Example: Determinant of a Triangular Matrix
• Hence, the determinant of a diagonal matrix is just the product of its diagonal entries.

27. General Properties of Determinants27
November November 2018
General Properties of Determinants
Theorem 1:
• (a) can be realized just by looking at the checkerboard mentioned above.
A=[ ; ; ]; det(A) = - 60
A=[ ; ; ]; det(A) = 60
A=[ ; ; ]; det(A) = - 60
• (b) points us to an attractive way of finding determinants: by reduction to triangular form.
• from (c): det⁡(𝑐𝑨) = 𝑐𝑛 det⁡(𝑨)

28. General Properties of Determinants28
November November 2018
General Properties of Determinants
Theorem 2:
• (f) can be proven from Theorem 1 and (e).
• from (f) & (b): a matrix with rank < n has det = zero.
eg:

29. Example: Finding Determinants by Reduction to Triangular Form29
November November 2018
Example: Finding Determinants by Reduction to Triangular Form

30. Example: Finding Determinants by Reduction to Triangular Form30
November November 2018
Example: Finding Determinants by Reduction to Triangular Form

31. Example: Finding Determinants by Reduction to Triangular Form31
November November 2018
Example: Finding Determinants by Reduction to Triangular Form

32. Example: Finding Determinants by Reduction to Triangular Form32
November November 2018
Example: Finding Determinants by Reduction to Triangular Form

33. Example: Finding Determinants by Reduction to Triangular Form33
November November 2018
Example: Finding Determinants by Reduction to Triangular Form