Chapter 3 Solution of Algebraic Equations 1 ChE 401: Computational Techniques for Chemical Engineers Fall 2009/2010 DRAFT SLIDES

1.Basic definitions 2.Consistency of equations and existence of solutions 3.Gaussian Elimination 4.LU Factorization 5.Gauss-Seidel Method Chapter 1 Lecture Outline 2 Chapter 3 DRAFT SLIDES

Chapter 1 Basic definitions 3 Chapter 3  Representation of linear systems DRAFT SLIDES

Chapter 1 4 Chapter 3 Matrix rank The rank r of matrix A is defined as the order of the largest nonsingular square matrix within A. OR The rank of a matrix is the maximum number of independent rows (or, the maximum number of independent columns). Singular matrix is a matrix that has zero determinant DRAFT SLIDES

Chapter 1 5 Chapter 3 Matrix rank Example 1: consider the following (3  4) matrix: There are four sub-matrices of order (3  3): det = 0 r = 2 DRAFT SLIDES

Chapter 1 6 Chapter 3 DRAFT SLIDES

Chapter 1 7 Chapter 3 Consistency of equations and existence of solutions The solution of a system of linear equations may or may not exist, and it may or may not be unique. Consider the following general system. N unknowns are related by M equations DRAFT SLIDES

Chapter 1 8 Chapter 3 Consistency of equations and existence of solutions M = N has unique solution if - no row is a linear combination of the others (row degeneracy), or - no column is a linear combination of the others (column degeneracy) Nonsingular matrix has a unique solution Mathematically these statements are exact, but, Numerically….. DRAFT SLIDES

Chapter 1 9 Chapter 3 Consistency of equations and existence of solutions: numerical issues If a system is too close to linear dependence …  an algorithm may fail altogether to get a solution  round off errors can produce apparent linear dependence at some point in the solution process f 1 = t+1 f 2 = 0.95t  The numerical procedure will fail totally DRAFT SLIDES

Chapter 1 10 Chapter 3 Consistency of equations and existence of solutions: numerical issues If a system is too close to linear dependence …  an algorithm may still work but produce nonsense  accumulated round of errors can swamp the solution - particularly in close-to-singular systems - particularly if N is too large  not algorithmic failure, but answer is (wildly) incorrect  Error can be confirmed by direct substitution in original equations How can we check for the error in answers in this case? DRAFT SLIDES

Chapter 1 11 Chapter 3 Consistency of equations and existence of solutions Under-determined systems…..  M<N OR M = N with degenerate equations  Fewer equations than unknowns  May be no solution, OR  May be an infinite number of solutions i.e. subspaces of solutions - arbitrary values must be assigned to (N-M) unknowns. - changing the values of (N-M) unknowns, results in new values for the rest of unknowns. - singular value decomposition is a powerful technique DRAFT SLIDES

Chapter 1 12 Chapter 3 Consistency of equations and existence of solutions Over-determined systems…..  M>N AND no degenerate  More equations than unknowns  Generally there is no solution  The best compromise solution is sought - closest to satisfy all equations - requires quantification of “closeness” to correct solution - most often: sum of squares of differences between left and right hand sides is minimized  linear least squares problem DRAFT SLIDES

Chapter 1 13 Chapter 3 Consistency of equations and existence of solutions: second approach… The solution of a system of linear equations may or may not exist, and it may or may not be unique. Existence of solutions can be determined by comparing the rank of the Matrix A with the rank of the augmented matrix A a. N unknowns are related by M equations DRAFT SLIDES

Chapter 1 14 Chapter 3 Consistency of equations and existence of solutions The set of equations has a solution if, and only if, the rank of the augmented matrix is equal to the rank of the coefficient matrix. If in addition: rank (A) = N  unique solution rank (A) < N & rank (A) = rank (A a )  Infinite number of solutions rank (A) < N & rank (A) < rank (A a )  No solution DRAFT SLIDES

Chapter 1 15 Chapter 3 Some examples Solve the following set of 2 equations in 2 unknowns for x and y. DRAFT SLIDES

Chapter 1 16 Chapter 3 DRAFT SLIDES

Chapter 1 17 Chapter 3 Some examples Solution DRAFT SLIDES

Direct methods: It consists of techniques that do not require iterations to find the solution. Most direct methods involve manipulations of the matrix (A)and vector b to obtain easy to solve equations. Chapter 1 Solution of Algebraic Equations 18 Chapter 3 Indirect (Iterative) methods: These methods start with initial approximation (guess) x o to the solution x and generate a sequence of vectors x k that converge to the solution vector. These techniques are efficiently used to solve large systems where computer storage and computational time are important considerations. DRAFT SLIDES

- It systematically eliminates unknowns from the problem till only one equation and unknown are left. - The value of this unknown is determined and the remaining unknowns are calculated in turn. Chapter 1 Gaussian Elimination (GE) 19 Chapter 3 Gaussian elimination is very efficient method in solving large sets of linear equations. For large n, it requires n 3 /2 calculations to get the solution. DRAFT SLIDES

Consider: Chapter 1 Gaussian Elimination 20 Chapter 3 We use row operations to simplify the system. e.g. eliminate element A 21 by subtracting A 21 /A 11 = d 21 times row 1 from row 2. Pivot row Pivot element DRAFT SLIDES A ’ 22 A ’ 23 A ’ 32 A ’ 33 A ” 33

Chapter 1 Gaussian Elimination (A 21 ) 21 Chapter 3 DRAFT SLIDES

Chapter 1 Gaussian Elimination (A 31 ) 22 Chapter 3 DRAFT SLIDES

Chapter 1 Gaussian Elimination (A 32 ) 23 Chapter 3 DRAFT SLIDES

Chapter 1 Backsubstitutation 24 Chapter 3 DRAFT SLIDES

Chapter 1 25 Chapter 3 Example  -4/3*-4 8/ /3 DRAFT SLIDES

Chapter 1 26 Chapter 3 Example -1/3* -1 2/ /3 DRAFT SLIDES

Chapter 1 27 Chapter 3 Example -5/14* 0 -5/3 30/14 130/42 DRAFT SLIDES

Chapter 1 28 Chapter 3 Example DRAFT SLIDES

Chapter 1 29 Chapter 3 DRAFT SLIDES

Chapter 1 30 Chapter 3 Example 2.1 in the textbook to be discussed…. DRAFT SLIDES

Chapter 1 31 Chapter 3 Example: 2.2  0 PIVOTINGPARTIAL There is no restriction upon which equation is placed first or which variable is eliminated first…. DRAFT SLIDES

Chapter 1 32 Chapter 3 Use Gaussian elimination to solve the following set of equations 2-digit arithmetic d 21 = 100 a 21 = 0 a 22 =  100 b 2 =  -100 x 2 = -1 x 1 = ( )/0.01 = 0 Exact solution… x 2 = x 1 = % error 100 % error DRAFT SLIDES

Chapter 1 33 Chapter 3 Try this…… 2-digit arithmetic d 21 = 0.01 a 22 =  -1.0 b 2 =  -1.0 x 2 = -1 x 1 =  1. 0 Exact solution… x 2 = x 1 = % error USE PIVOTING What did you learn ? DRAFT SLIDES

Chapter 1 34 Chapter 3 What about this??? Multiply the first equation by 200, what do you find? TRY IT BY YOURSELF DRAFT SLIDES

LU factorization (without pivoting) Let: A = L.U with L unit lower triangular, U upper triangular, i.e. Chapter 1 LU Factorization 35 Chapter 3 DRAFT SLIDES

LU factorization (without pivoting) This factorization can be used to solve the following linear set A.x = (L.U).x = L.(U.x) = L.y = b Such that L.y = b (forward substitution) U.x = y (backward substitution) Chapter 1 LU Factorization 36 Chapter 3 How many set of linear equations do we have now? What is the advantage of breaking up one linear set into two successive ones? DRAFT SLIDES

Algorithm…. This method is called Doolittle’s factorization Partition A, L, U as block matrices: with L 22 unit-lower triangle and U 22 upper triangle Chapter 1 LU Factorization 37 Chapter 3 DRAFT SLIDES

Determine L, U from A = L.U, i.e. this gives u 11 = a 11, U 12 = A 12, L 21 = (1/a 11 )A 21 L 22 U 22 = A 22 – L 21 U 12 Chapter 1 LU Factorization 38 Chapter 3 DRAFT SLIDES

Example…. Factorize matrix A Chapter 1 LU Factorization 39 Chapter 3 Solution… DRAFT SLIDES

Chapter 1 40 Chapter 3 Solution… Step 1: First row of U and first column of L Step 2: Find l 32, u 22, u 23 and u 33 L 22 U 22 = A 22 – L 21 U 12 = - DRAFT SLIDES

Chapter 1 41 Chapter 3 Solution… Step 2: Find l 32, u 22, u 23 and u 33 = - = DRAFT SLIDES

Note….  Re-solve the previous example using GE method Chapter 1 LU Factorization 42 Chapter 3 What are the values for d ij ? d 21 = 1/2 d 31 = 6/8 d 32 = 11/16 Compare DRAFT SLIDES

Note….  Re-solve the previous example using GE method Chapter 1 LU Factorization 43 Chapter 3 After elimination steps we got Compare DRAFT SLIDES

Note….  Solving a set of equations using LU decomposition takes the same number of calculations as Gaussian elimination since both methods are essentially equivalent. Why to use this method? Chapter 1 LU Factorization 44 Chapter 3 1. The solution of a triangular set of equations is quite trivial, forward substitution is used to find y elements and vector x is obtained using back substitution scheme DRAFT SLIDES

Note….  Solving a set of equations using LU decomposition takes the same number of calculations as Gaussian elimination since both methods are essentially equivalent. Why to use this method? Chapter 1 LU Factorization 45 Chapter 3 2. Once A is transformed into L and U, it can be used with any right-hand-side vector, b. DRAFT SLIDES

MATLAB has built in routines for solving linear systems. For LU decomposition, use the command “lu”. Consider the previous example Chapter 1 LU Factorization 46 Chapter 3 A =[8 2 9;4 9 4;6 7 9]; [L,U]=lu(A) b=[1;2;3]; Y=L \ b; X=U\Y DRAFT SLIDES

Example 2.4, p.26 Chapter 1 47 Chapter 3 DRAFT SLIDES

Example 2.4, p.26 Chapter 1 48 Chapter 3 [ ] x b A DRAFT SLIDES

Example 2.4, p.26 Chapter 1 49 Chapter 3 For MATLAB code, see m-file called Example 24 DRAFT SLIDES

Chapter 1 50 Chapter 3 Example 2.3 in the textbook to be discussed…. DRAFT SLIDES

A three-stages absorption system, shown in the figure below, is described by the following set of equations: Chapter 1 Example…. 51 Chapter 3 Solve for the values of x i and y i, for the following data: x f = 0.01, y f = 0.06, L = 40.8 lb/min, G = 66.7 lb/min, a=0.72. Note that y i = ax i 51DRAFT SLIDES

Chapter 1 Solution 52 Chapter 3 Use MATLAB…. 52DRAFT SLIDES Write your code

Chapter 1 53 Chapter 3 DRAFT SLIDES System Condition Condition number ~ 10 4,It is ill-conditioned system Determinant = 1  10 -4

Chapter 1 Gauss-Seidel Method 54 Chapter 3 Why to use this method? -It requires fewer calculations than GE for very large systems of linear equations or for linear equations with a sparse coefficient matrix which is not banded. -It is an efficient method for the solution of ill-conditioned set of linear equations. -If the physics of the problem are understood, a close initial guess can be made, decreasing the number of iterations needed. DRAFT SLIDES An iterative approach

Chapter 1 Gauss-Seidel Method 55 Chapter 3 Basic Procedure: -Algebraically solve each linear equation for x i -Assume an initial guess solution array -Solve for each x i -Check for error after each iteration to check if error is within a pre-specified tolerance. -Repeat until error is within the tolerance. DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 56 Chapter 3 Algorithm A set of n equations and n unknowns:.. If the diagonal elements are non- zero Rewrite each equation solving for the corresponding unknown For example: First equation, solve for x 1 n equation, solve for x n DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 57 Chapter 3 Algorithm Rewriting each equation From Equation 1 From equation 2 From equation n-1 From equation n DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 58 Chapter 3 Algorithm General Form for any row ‘i’ DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 59 Chapter 3 Solve for the unknowns Assume an initial guess for [X] Use rewritten equations to solve for each value of x i. Remember to use the most recent value of x i. DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 60 Chapter 3 Check for the tolerance using a pre-defined criterion such as The iterations are stopped when the error is less than a pre-specified tolerance for all unknowns. DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 61 Chapter 3 Example…. Consider the following set of linear equations: DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 62 Chapter 3 Applying the initial guess and solving for x i Initial Guess When solving for x 2, how many of the initial guess values were used? DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 63 Chapter 3 Finding the percentage relative error At the end of the first iteration The maximum percentage relative error is % DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 64 Chapter 3 Iteration #2 Using from iteration #1 the values of a i are found: DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 65 Chapter 3 Finding the percentage relative error At the end of the second iteration The maximum percentage relative error is % DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 66 Chapter 3 Iteration x1x1 x2x2 x3x − − − − − − − − − −14440 −60072 − Repeating more iterations, we got Notice – The relative errors are not decreasing Also, the solution is not converging to the true solution DRAFT SLIDES

Chapter 1 Gauss-Seidel Method 67 Chapter 3 DRAFT SLIDES What is the problem? This example illustrates a drawback of the Gauss-Siedel method: not all systems of equations will converge. Is it fixable? One class of system of equations always converges: One with a diagonally dominant coefficient matrix. Diagonally dominant: A in A.X = b is diagonally dominant if:

Chapter 1 Gauss-Seidel Method 68 Chapter 3 DRAFT SLIDES Which coefficient matrix is diagonally dominant? Most physical systems do result in simultaneous linear equations that have diagonally dominant coefficient matrices.

Chapter 1 Gauss-Seidel Method 69 Chapter 3 DRAFT SLIDES Example 2 Given the system of equations With an initial guess of The coefficient matrix is: Will the solution converge using the Gauss-Siedel method?

Chapter 1 Gauss-Seidel Method 70 Chapter 3 DRAFT SLIDES Checking if the coefficient matrix is diagonally dominant The inequalities are all true; therefore the solution should converge using the Gauss-Seidel Method

Chapter 1 Gauss-Seidel Method 71 Chapter 3 DRAFT SLIDES Rewriting each equationWith an initial guess of

Chapter 1 Gauss-Seidel Method 72 Chapter 3 DRAFT SLIDES The absolute relative approximate error The maximum percentage relative error after the first iteration is 100%

Chapter 1 Gauss-Seidel Method 73 Chapter 3 DRAFT SLIDES Iterationx1x1 x2x2 x3x Repeating more iterations, the following values are obtained The solution obtained Note that the exact solution is.

Chapter 1 Gauss-Seidel Method 74 Chapter 3 DRAFT SLIDES Example 3 Given the system of equations With an initial guess of

Chapter 1 Gauss-Seidel Method 75 Chapter 3 DRAFT SLIDES Iterationx1x1 x2x2 x3x − − − ×10 5 −2.0579× − − × − − ×10 5 −4.8653× Conducting six iterations, the following values are obtained The values are not converging? Your comments……..

Chapter 1 Gauss-Seidel Method 76 Chapter 3 DRAFT SLIDES Example 3 Example 2 If a system of linear equations is not diagonally dominant, check to see if rearranging the equations can form a diagonally dominant matrix.