MA/CS 375 Fall MA/CS 375 Fall 2003 Lecture 19

MA/CS 375 Fall Matrices and Systems

MA/CS 375 Fall Upper Triangular Systems

MA/CS 375 Fall Upper Triangular Systems Volunteer: tell us how to solve this? (hint: this is not difficult)

MA/CS 375 Fall Example: Upper Triangular Systems

MA/CS 375 Fall Upper Triangular Systems 1) Solve the N th equation 2) Use the value from step 1 to solve the (N-1) th equation 3) Use the values from steps 1 and 2 to solve the (N-2) th equation 4) keep going until you solve the whole system.

MA/CS 375 Fall Backwards Substitution

MA/CS 375 Fall Matlab Code for Backward Substitution

MA/CS 375 Fall Testing Backwards Substitution

MA/CS 375 Fall Finite Precision Effects Notice that x(1) depends on the values of x(2),x(3),…,x(N) Remembering that round off rears its ugly head when we deal with (small+large) numbers Volunteer to design a U matrix and b vector, which do not give good answers when used in this algorithm. (goal is to give you some intuition for cases which will break an algorithm)

MA/CS 375 Fall Team Exercise Who can build a “reasonable”, non- singular, U matrix and b vector which has the worst answer for U\b 10 Minutes only

MA/CS 375 Fall LU Factorization By now you should be persuaded that it is not difficult to solve a problem of the form Lx=b or Ux=b using forward/backward elimination Suppose we are given a matrix A and we are able to find a lower triangular matrix L and an upper triangular matrix U such that: A = LU Then to solve Ax=b we can solve two simple problems: Ly = b Ux = y (sanity check LUx = L(Ux) = Ly =b)

MA/CS 375 Fall Example LU Factorization Suppose we are given: Then we can write A = LU where: Let’s check that:

MA/CS 375 Fall LU Factorization (in words) Construct a sequence of multiplier matrices (M 1, M 2, M 3, M 4,…., M N-1 ) such that: (M N-1 …. M 4 M 3 M 2 M 1 )A is upper triangle. A suitable candidate for M 1 is the matrix ( 4 by 4 case):

MA/CS 375 Fall Check to See What M 1 Does to A

MA/CS 375 Fall M 1 A Has Zeros Below The Diagonal

MA/CS 375 Fall LU Factorization cont. A suitable candidate for M 2 is the matrix ( 4 by 4 case):

MA/CS 375 Fall LU Factorization cont. A suitable candidate for M 3 is the matrix ( 4 by 4 case):

MA/CS 375 Fall LU Factorization cont. So in this case: U=(M 3 M 2 M 1 )A is upper triangle Each of the M matrices is lower triangle The inverse of M 2 looks like: Using the fact that the inverse of the each M matrix just requires us to negate the below diagonal terms A = (M 3 M 2 M 1 ) -1 U = (M 1 ) -1 (M 2 ) -1 (M 3 ) -1 U Define L = (M 1 ) -1 (M 2 ) -1 (M 3 ) -1 and we are done since the product of lower matrices is a lower matrix

MA/CS 375 Fall Matlab Routine For LU Factorization

MA/CS 375 Fall Matlab Routine For LU Factorization Build L and U A is modified in place Total cost O(N 3 ) ( we can get this by noticing the triply nested loop)

MA/CS 375 Fall Matlab Routine For LU Factorization And Solution of Ax=b Build rhs Solve Ly=b Solve Ux=y

MA/CS 375 Fall Team Exercise 1)Code up the factorization 2)Test your code to make sure it gives the correct answer as Matlab Make sure abs(LU – Aorig) is small Make sure abs(x - Aorig\b) is small 3)Make sure the routine actually finished without the break. 4)Time the code for an N=400 case 5)Compare with the timings for [Lmat,Umat] = lu(Aorig); ASK IF YOU DO NOT UNDERSTAND ANY PART OF THIS!!

MA/CS 375 Fall When Basic LU Does Not Work Remember that we are required to compute 1/A(i,i) for all the successive diagonal terms What happens when we get to the i’th diagonal term and this turns out to be 0 or very small ?? At any point we can swap the i’th row with one lower down which does not have a zero (or small entry) in the i’th column. This is known as partial pivoting.

MA/CS 375 Fall Partial Pivoting Remember before that we constructed a set of M matrices so that is upper triangle Now we construct a sequence of M matrices and P matrices so that: is upper triangle. (M N-1 …. M 4 M 3 M 2 M 1 )A (M N-1 P N-1 …. M 4 P 4 M 3 P 3 M 2 P 2 M 1 P 1 )A

MA/CS 375 Fall LU with Partial Pivoting

MA/CS 375 Fall LU with Partial Pivoting 1)find pivot 2)swap rows 3)Do elimination

MA/CS 375 Fall LU with Partial Pivoting 1)build matrix A 2)call our LU routine 3)check that: A(piv,:) = L*U 4)Looks good

MA/CS 375 Fall Team Exercise Make sure you all have a copy of the.m files on your laptops (courtesy of Coutsias) Locate GEpiv For N=100,200,400,600 –build a random NxN matrix A –time how long GEpiv takes to factorize A end Plot the cputime taken per test against N using loglog On the same graph plot N^3 using loglog Observation