Linear Systems of Equations Ax = b Marco Lattuada Swiss Federal Institute of Technology - ETH Institut für Chemie und Bioingenieurwissenschaften ETH Hönggerberg/ HCI F135 – Zürich (Switzerland) Marco Lattuada Swiss Federal Institute of Technology - ETH Institut für Chemie und Bioingenieurwissenschaften ETH Hönggerberg/ HCI F135 – Zürich (Switzerland)

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 2 Definition of the Problem We want to solve: where x is the vector of the unknowns, while A and b are given. Hypotheses: 1.The number of equations is equal to the number of unknowns (that is, A is a square matrix) 1.The coefficients of A, b and x are real 2.The solution of the system exists and it is unique We want to solve: where x is the vector of the unknowns, while A and b are given. Hypotheses: 1.The number of equations is equal to the number of unknowns (that is, A is a square matrix) 1.The coefficients of A, b and x are real 2.The solution of the system exists and it is unique A -1 exists A is not singular A's columns are linearly independent A's lines are linearly independent det(A) is non-zero rank(A) is equal to n Ax = 0 only if x is a null vector

Analytical Approach Cramer’s rule (1750): The solution of a system of equations: Is given by: where A i is defined as follows: Cramer’s rule (1750): The solution of a system of equations: Is given by: where A i is defined as follows: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 3 b replaces the i th column

Calculation of the determinant How to compute the determinant of a square matrix? Laplace formula (1772): where C i,j is the cofactor of element a i,j. The cofactor C i,j is the determinant of the submatrix obtained by removing the i th row and the j th column of the matrix, multiplied by (-1) i+j : How to compute the determinant of a square matrix? Laplace formula (1772): where C i,j is the cofactor of element a i,j. The cofactor C i,j is the determinant of the submatrix obtained by removing the i th row and the j th column of the matrix, multiplied by (-1) i+j : Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 4 No i th row No j th column

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 5 Numerical approach: Gauss Elimination Method Let us consider the system: Let us consider the following operations: 1.I multiply one line by a constant 2.I substitute one line with a linear combination of the others 3.I operate a permutation of the lines The result does not change Let us consider the system: Let us consider the following operations: 1.I multiply one line by a constant 2.I substitute one line with a linear combination of the others 3.I operate a permutation of the lines The result does not change

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 6 Gauss Elimination Method Numerical Example Multiply by -3Multiply by -3 Sum it to 1st lineSum it to 1st line Multiply by -3Multiply by -3 Sum it to 1st lineSum it to 1st line Multiply by -4Multiply by -4 Sum it to 2nd lineSum it to 2nd line Triangular System

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 7 Gauss Elimination Method General Case I want to replace a 21 with a zero I define the multiplier l 21 : Note that:

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 8 Gauss Elimination Method Gauss Transformation Matrix where: Solution:

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 9 Gauss Elimination Method Numerical Example n(n-1) operations (flops) (n-1)(n-2) operations (flops) Total number of operations required

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 10 Gauss Transformation Method Let us change our point of view! can be used to transform A Gauss Elimination Method 1.Changes the matrix A 2.Needs the coefficient vector b 3.Must re-run the method if b is changed

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 11 Gauss Transformation Method Properties The final matrix A is a right triangular matrix The matrix M is a left triangular matrix The inverse of M is also a left triangular matrix The matrix L = M -1 has the simple form: Properties The final matrix A is a right triangular matrix The matrix M is a left triangular matrix The inverse of M is also a left triangular matrix The matrix L = M -1 has the simple form:

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 12 Consider the following expression: Let us multiply by L = M -1 both sides: Consider the following expression: Let us multiply by L = M -1 both sides: LR (LU) Factorization Right triangular

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 13 Starting matrix A is transformed (factorized) as: Let us solve a linear system with a generic vector b: Starting matrix A is transformed (factorized) as: Let us solve a linear system with a generic vector b: LR Factorization 1.For every vector b, two simple triangular systems must be solved without factorizing again 2.The matrices LR can be stored using the elements of A 3.If A is modified, it is often possible to modify L and R accordingly without factorizing

Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 14 Starting matrix: Consider the following system: Consider the following similar system: Starting matrix: Consider the following system: Consider the following similar system: Problems of Gaussian Elimination and LR Pivot value must be ≠ 0 a 11 = 0  I switch the lines  x 1 = 1 and x 2 = 1 Manual LR Factorization without pivoting