1. MA/CS 375 Fall 20031 MA/CS 375 Fall 2003 Lecture 19

2. MA/CS 375 Fall 20032 Matrices and Systems

3. MA/CS 375 Fall 20033 Upper Triangular Systems

4. MA/CS 375 Fall 20034 Upper Triangular Systems Volunteer: tell us how to solve this? (hint: this is not difficult)

5. MA/CS 375 Fall 20035 Example: Upper Triangular Systems

6. MA/CS 375 Fall 20036 Upper Triangular Systems 1) Solve the N th equation 2) Use the value from step 1 to solve the (N-1) th equation 3) Use the values from steps 1 and 2 to solve the (N-2) th equation 4) keep going until you solve the whole system.

7. MA/CS 375 Fall 20037 Backwards Substitution

8. MA/CS 375 Fall 20038 Matlab Code for Backward Substitution

9. MA/CS 375 Fall 20039 Testing Backwards Substitution

10. MA/CS 375 Fall 200310 Finite Precision Effects Notice that x(1) depends on the values of x(2),x(3),…,x(N) Remembering that round off rears its ugly head when we deal with (small+large) numbers Volunteer to design a U matrix and b vector, which do not give good answers when used in this algorithm. (goal is to give you some intuition for cases which will break an algorithm)

11. MA/CS 375 Fall 200311 Team Exercise Who can build a “reasonable”, non- singular, U matrix and b vector which has the worst answer for U\b 10 Minutes only

12. MA/CS 375 Fall 200312 LU Factorization By now you should be persuaded that it is not difficult to solve a problem of the form Lx=b or Ux=b using forward/backward elimination Suppose we are given a matrix A and we are able to find a lower triangular matrix L and an upper triangular matrix U such that: A = LU Then to solve Ax=b we can solve two simple problems: Ly = b Ux = y (sanity check LUx = L(Ux) = Ly =b)

13. MA/CS 375 Fall 200313 Example LU Factorization Suppose we are given: Then we can write A = LU where: Let’s check that:

14. MA/CS 375 Fall 200314 LU Factorization (in words) Construct a sequence of multiplier matrices (M 1, M 2, M 3, M 4,…., M N-1 ) such that: (M N-1 …. M 4 M 3 M 2 M 1 )A is upper triangle. A suitable candidate for M 1 is the matrix ( 4 by 4 case):

15. MA/CS 375 Fall 200315 Check to See What M 1 Does to A

16. MA/CS 375 Fall 200316 M 1 A Has Zeros Below The Diagonal

17. MA/CS 375 Fall 200317 LU Factorization cont. A suitable candidate for M 2 is the matrix ( 4 by 4 case):

18. MA/CS 375 Fall 200318 LU Factorization cont. A suitable candidate for M 3 is the matrix ( 4 by 4 case):

19. MA/CS 375 Fall 200319 LU Factorization cont. So in this case: U=(M 3 M 2 M 1 )A is upper triangle Each of the M matrices is lower triangle The inverse of M 2 looks like: Using the fact that the inverse of the each M matrix just requires us to negate the below diagonal terms A = (M 3 M 2 M 1 ) -1 U = (M 1 ) -1 (M 2 ) -1 (M 3 ) -1 U Define L = (M 1 ) -1 (M 2 ) -1 (M 3 ) -1 and we are done since the product of lower matrices is a lower matrix

20. MA/CS 375 Fall 200320 Matlab Routine For LU Factorization

21. MA/CS 375 Fall 200321 Matlab Routine For LU Factorization Build L and U A is modified in place Total cost O(N 3 ) ( we can get this by noticing the triply nested loop)

22. MA/CS 375 Fall 200322 Matlab Routine For LU Factorization And Solution of Ax=b Build rhs Solve Ly=b Solve Ux=y

23. MA/CS 375 Fall 200323 Team Exercise 1)Code up the factorization 2)Test your code to make sure it gives the correct answer as Matlab Make sure abs(LU – Aorig) is small Make sure abs(x - Aorig\b) is small 3)Make sure the routine actually finished without the break. 4)Time the code for an N=400 case 5)Compare with the timings for [Lmat,Umat] = lu(Aorig); ASK IF YOU DO NOT UNDERSTAND ANY PART OF THIS!!

24. MA/CS 375 Fall 200324 When Basic LU Does Not Work Remember that we are required to compute 1/A(i,i) for all the successive diagonal terms What happens when we get to the i’th diagonal term and this turns out to be 0 or very small ?? At any point we can swap the i’th row with one lower down which does not have a zero (or small entry) in the i’th column. This is known as partial pivoting.

25. MA/CS 375 Fall 200325 Partial Pivoting Remember before that we constructed a set of M matrices so that is upper triangle Now we construct a sequence of M matrices and P matrices so that: is upper triangle. (M N-1 …. M 4 M 3 M 2 M 1 )A (M N-1 P N-1 …. M 4 P 4 M 3 P 3 M 2 P 2 M 1 P 1 )A

26. MA/CS 375 Fall 200326 LU with Partial Pivoting

27. MA/CS 375 Fall 200327 LU with Partial Pivoting 1)find pivot 2)swap rows 3)Do elimination

28. MA/CS 375 Fall 200328 LU with Partial Pivoting 1)build matrix A 2)call our LU routine 3)check that: A(piv,:) = L*U 4)Looks good

29. MA/CS 375 Fall 200329 Team Exercise Make sure you all have a copy of the.m files on your laptops (courtesy of Coutsias) Locate GEpiv For N=100,200,400,600 –build a random NxN matrix A –time how long GEpiv takes to factorize A end Plot the cputime taken per test against N using loglog On the same graph plot N^3 using loglog Observation