1. MA2213 Lecture 5 Linear Equations (Direct Solvers)

2. Systems of Linear Equations p. 243-248 Occur in a wide variety of disciplines Mathematics Statistics Physics ChemistryBiology Economics Sociology Psychology Archaeology Geology AstronomyAnthropology Engineering Management Business Medicine Finance

3. Matrix Form for a system of linear equations coefficient matrix (solution) column vector (right) column vector

4. Linear Equations in Mathematics Numerical Analysis Geometry Interpolation Least Squares Quadrature Algebra find intersection of lines or planes partial fractions Coefficient Matrix Vandermonde (for polyn. interp.) or Gramm Transpose of Vandermonde Lec 4 vufoil 13 (to compute weights)

5. Matrix Arithmetic p. 248-264 Matrix Inverse Matrix Multiplication Identity Matrix Theorem 6.2.6 p. 255 A square matrix has an inverse iff (if and only if) its determinant is not equal to zero.

6. Solution of (this means exists and is unique. multiplication is associative for nonsingular Proof Remark In MATLAB use: x = A \ b;

7. Column Rank of a Matrix Definition The column rank of a matrix dimension of the subspace of spanned by the column vectors of Remark maximal number of linearly independent column vectors Question is the of

8. Row Rank of a Matrix Definition The row rank of a matrix is the dimension of the subspace of spanned by the row vectors of Remark maximal number of linearly independent row vectors of Question

9. A Matrix Times a Vector has solution iff b is a linear combination of columns of A The equation

10. Existence of Solution in General The linear equation has a solution if and only if IS SINGULAR! EVEN IF Example this has a solution iff then it has an infinite number of solutions called Augmented matrix p. 265

11. Computing the Column and Row Ranks The ranks of a matrix can be computed using a sequence of elementary row operations p. 253-254. i. Interchange two rows ii. Multiply a row by a nonzero scalar iii. Add a nonzero multiple of one row to another row Question Show that each of the ERO i, ii, iii has an inverse ERO i, ii, iii.

12. Elementary Row Operations can be performed on the left by by multiplying on a matrix nonsingular matrices

13. Invariance of Row Rank Under ERO Theorem 1. If is an ERO matrix, then Proof Clearly, interchanging two rows and multiplying a row by a nonzero scalar does not change the row rank. Finish the proof by showing that adding a multiple of any row to another row does not change the row rank. and Remark Clearly the row rank of a matrix is invariant under sequence of ERO’s.

14. Matrix Multiplication

15. Invariance of Column Rank under ERO Theorem 2 If is nonsingular then Proof It suffices to show that for a set are linearly dependent iff the set of ofof column vectors are linearly dependent. Show why it suffices and then show it. Hint: prove and column vectorsof

16. Row Echelon Matrices Definition A matrix an row echelon matrix if i. the nonzero rows come first ii. the first nonzero element in each row =1 (called a pivot) has all zeros below it is called iii. each pivot lies to the right of the pivot in the row above

17. Row Echelon Matrices These three properties produce a staircase pattern in the matrix below Question Where are the pivots ?

18. Row Rank of an Row Echelon Matrix equals the number of nonzero rows. Question What is the rank of this matrix ? Prove this by showing that the rows must be linearly independent. Hint : use pivots.

19. Col. Rank of a Row Echelon Matrix equals the number of nonzero rows. Question Show this by showing that the col. vectors that contain pivots form a basis for the space spanned by col. vectors. Hint: do elem. col. operations on the matrix above.

20. Reduction to Row Echelon Form Theorem 3 For every matrix there exists a nonsingular matrix is an echelon matrix. such that Furthermore, the matrixis a product where each is an ERO matrix. Application of the sequence of ERO’s is called reduction to row echelon form. Proof Based on Gaussian elimination.

21. Row Rank = Column Rank Theorem 4 For every matrix Proof. Theorem 3 implies that there exists a product of ERO matrices such thatis a row echelon matrix. Theorems 1 implies that and theorems 2 implies that Sinceis a row echelon matrix, hence

22. Applications of Row Echelon Reduction The linear equation iff the last nonzero row of the reduced Example has a solution has its pivot NOT in the last column. Hence the condition above is satisfied iff

23. Applications of Row Echelon Reduction A basis of column vectors for a matrix can be obtained by first computing the reduction then choosing the column vectors that form a basis for the space spanned by the column that contain the pivots. Then the vectors are column vectors of vectors of

24. Generalities on Gaussian Elimination Gaussian elimination is the process of reducing a matrix to row echelon form through a sequence of ERO’s. It can also be used to solve a system of linear equations The final step of solving a system of equations after the augmented matrix has been reduced is called back substitution, this process is related to elementary column operations and will be addressed in the homework. It is ‘best’ taught through showing examples. We will show how to solve a system of linear equations using Gaussian elimination, it will become obvious how to use Gaussian elimination for reduction.

25. Gaussian Elimination (p. 264-269) Case 1. The equations for this matrix are Question How do we use the nonsingular assumption? therefore, if A is nonsingular then Question What type of matrix is this ?

26. Back Substitution Case 2. A nonsingular  solution by back-substitution p. 265 Question How do we use the nonsingular assumption? Question What is this matrix called ? Question What are the associated equations ? Question Why is this method called back-substitution ?

27. Gaussian Elimination on Equations Case 3. Apply elementary row operations on equations to to obtain equations with an upper triangular matrix Question How can we solve these equations ?

28. Gaussian Elimination on Augmented Matrix

29. Gaussian Elimination Question What is the solution ?

30. Partial Pivoting p. 270-273 the integer that gives For the j-th column in Gaussian elimination compute then perform the row interchange Read p. 273-276 about how Gaussian elimination can be used to compute the inverse of a matrix.

31. LU Decomposition p. 283-285 To solve where Then for each b use forward substitution to solve L y = b then use backward substitution to solve U x = y. first compute the factorization for many values of with same

32. LU Decomposition Algorithm Algorithm Step 1 Step 2 for r = 2,…,n do Question How many operations does this require ?

33. Homework Due Tutorial 3 Question 1. Prove that the row rank of an row echelon matrix equals the number of nonzero rows. Question 2. Prove that the column rank of an row echelon matrix equals the number of nonzero rows by showing that the set of its column vectors having pivots is a maximal set of linearly independent column vectors. Question 3. Use Gaussian elimination to solve Question 4. Derive expressions for the entries of the L and U in the LU decomposition of a 3 x 3 matrix A. Question 5. Show how elementary column operations can be applied to a row echelon matrix M to obtain a row echelon matrix with exactly one 1 in each nonzero row. Use this to determine a basis for the space { x : Mx = 0 }.