Chapter 10 LU Decomposition

L and U Matrices Lower Triangular Matrix Upper Triangular Matrix

LU Decomposition Another method for solving matrix equations Idea behind the LU Decomposition - start with We know (because we did it in Gauss Elimination) we can write

LU Decomposition Assume there exists [L] Such that This implies

The Steps of LU Decomposition

LU Decomposition LU Decomposition Decomposition Methods (not unique) * Based on Gauss elimination *More efficient Decomposition Methods (not unique) * Doolittle decomposition lii = 1 * Crout decomposition uii = 1 (omitted) * Cholesky decomposition (for symmetric matrices) uii = lii

LU Decomposition Three Basic Steps (1) Factor (decompose) [A] into [L] and [U] (2) given {b}, determine {d} from [L]{d} = {b} (3) using [U]{x} = {d} and back-substitution, solve for {x} Advantage: Once we have [L] and [U], we can use many different {b}’s without repeating the decomposition process

LU Decomposition LU decomposition / factorization [ A ] { x } = [ L ] [ U ] { x } = { b } Forward substitution [ L ] { d } = { b } Back substitution [ U ] { x } = { d } Forward substitutions are more efficient than elimination

Simple Truss F45 4 5 F35 F14 F24 F25 1 3 H1 F12 F23 2 V1 V2 W

Exampe: Forces in a Simple Truss [A] depends on geometry only; but {b} varies with applied load

LU Factorization [ A ] = [ L ] [ U ] Gauss elimination [A]{x}={b} elimination steps need to be repeated for each different {b} LU factorization / decomposition [ A ] = [ L ] [ U ] does not involve the RHS {b} !!!

Doolittle LU Decomposition Doolittle Algorithm Get 1’s on diagonal of [L] (lii =1) Operate on rows and columns sequentially, narrowing down to single element Identical to LU decomposition based on Gauss elimination (see pp238-239), but different approach

Doolittle LU Decomposition

Doolittle LU Decomposition

Doolittle LU Decomposition

Algorithm of Doolittle Decomposition

Very efficient for large matrices ! Forward Substitution Once [L] is formed, we can use forward substitution instead of forward elimination for different {b}’s Very efficient for large matrices !

Identical to Gauss elimination Back Substitution Identical to Gauss elimination

Forward Substitution Example:

Back-Substitution

Forward and Back Substitutions Forward-substitution Back-substitution (identical to Gauss elimination) Error in textbook (p. 165)

Forward and Back Substitutions

Example: Forward and Back Substitutions » [L,U] = LU_factor(A); L = 1.0000 0 0 0 -1.0000 1.0000 0 0 0 0.5000 1.0000 0 6.0000 1.0000 14.0000 1.0000 U = 1 0 2 3 0 2 4 0 0 0 -1 4 0 0 0 -70 » x=LU_solve(L,U,b) x = -0.1857 0.2286 -0.1143 0.4714 (without pivoting) MATLAB M-file LU_factor Forward and back substitution

Pivoting in LU Decomposition Still need pivoting in LU decomposition Messes up order of [L] What to do? Need to pivot both [L] and a permutation matrix [P] Initialize [P] as identity matrix and pivot when [A] is pivoted  Also pivot [L]

LU Decomposition with Pivoting Permutation matrix [ P ] - permutation of identity matrix [ I ] Permutation matrix performs “bookkeeping” associated with the row exchanges Permuted matrix [ P ] [ A ] LU factorization of the permuted matrix [ P ] [ A ] = [ L ] [ U ] Solution [ L ] [ U ] {x} = [ P ] {b}

Permutation Matrix Bookkeeping for row exchanges Example: [ P1] interchanges row 1 and 3 Multiple permutations [ P ]

LU Decomposition with Pivoting Start with No need to consider {b} in decomposition

Forward Elimination Gauss elimination of first column Interchange rows 1 & 4 Gauss elimination of first column Save fi1 in the first column of [L]

Forward Elimination Gauss elimination of second column No interchange required Gauss elimination of second column Save fi2 in the second column of [L]

Forward Elimination Partial pivoting for [L] and [P] Interchange rows 3 &4 Partial pivoting for [L] and [P]

Forward Elimination Gauss elimination of third column Save fi3 in third column of [L]

LU Decomposition with Pivoting Gauss elimination with partial pivoting

LU Decomposition with Pivoting Forward substitution Back substitution

LU Decomposition with Pivoting partial pivoting

LU Decomposition with Pivoting » A=[1 0 2 3; -1 2 2 -3; 0 1 1 4; 6 2 2 4]; » b=[1 -1 2 1]'; » [L,U,P]=LU_pivot(A); L = 1.0000 0 0 0 -0.1667 1.0000 0 0 0.1667 -0.1429 1.0000 0 0 0.4286 0 1.0000 U = 6.0000 2.0000 2.0000 4.0000 0 2.3333 2.3333 -2.3333 0 0 2.0000 2.0000 0 0 0 5.0000 T1 = 6 2 2 4 -1 2 2 -3 1 0 2 3 0 1 1 4 T2 = T1 = [L][U] T2 = [P][A] Verify T1= T2

LU Decomposition with Pivoting » A=[1 0 2 3; -1 2 2 -3; 0 1 1 4; 6 2 2 4]; » b=[1; -1; 2; 1]; » [L,U,P]=LU_pivot(A); » Pb=P*b Pb = 1 -1 2 » x=LU_Solve(L,U,Pb) d = 1.0000 -0.8333 0.7143 2.3571 x = -0.1143 0.4714 0.2286 -0.1857 LU Decomposition [L][U]{x} = [P]{b} Forward Substitution Back Substitution

MATLAB’s Methods LU factorization [L,U] = lu (A) -- returns [L] and [U] [L, U, P] = lu (A) -- returns also the permutation matrix [P] Cholesky factorization: R = chol(A) -- [A] = [R][R] Determinant: det (A)

Forward and Back Substitutions LU Decomposition without Pivoting MATLAB function lu » A=[1 0 2 3; -1 2 2 -3; 0 1 1 4; 6 2 2 4]; » b=[1; -1; 2; 1]; » [L,U]=lu(A) L = 0.1667 -0.1429 1.0000 0 -0.1667 1.0000 0 0 0 0.4286 0 1.0000 1.0000 0 0 0 U = 6.0000 2.0000 2.0000 4.0000 0 2.3333 2.3333 -2.3333 0 0 2.0000 2.0000 0 0 0 5.0000 » L*U ans = 1 0 2 3 -1 2 2 -3 0 1 1 4 6 2 2 4 » d=L\b d = 1.0000 -0.8333 0.7143 2.3571 » x=U\d x = -0.1857 0.2286 -0.1143 0.4714 Forward and Back Substitutions LU Decomposition without Pivoting

Cholesky LU Factorization If [A] is symmetric and positive definite, it is convenient to use Cholesky decomposition. [A] = [L][L]T= [U]T[U] Cholesky factorization can be used for either symmetric or non-symmetric matrices No pivoting or scaling needed if [A] is symmetric and positive definite (all eigenvalues are positive) If [A] is not positive definite, the procedure may encounter the square root of a negative number

Cholesky LU Factorization For a general non-symmetric matrix For symmetric and positive definite matrices

Cholesky LU Decomposition

Example: Cholesky LU

Cholesky LU Factorization [A] = [U]T[U] = [U] [U] Recurrence relations

Script file for Cholesky Decomposition Symmetric Matrix L = U Compute only the upper triangular elements

» A=[9 -6 12 -3; -6 5 -9 2; 12 -9 21 0; -3 2 0 6] A = 9 -6 12 -3 -6 5 -9 2 12 -9 21 0 -3 2 0 6 » [L,U] = Cholesky(A) L = 3 0 0 0 -2 1 0 0 4 -1 2 0 -1 0 2 1 U = 3 -2 4 -1 0 1 -1 0 0 0 2 2 0 0 0 1 Symmetric [L] = [U]

MATLAB Function: chol Forward substitution Back substitution » U = chol(A) U = 3 -2 4 -1 0 1 -1 0 0 0 2 2 0 0 0 1 » U'*U ans = 9 -6 12 -3 -6 5 -9 2 12 -9 21 0 -3 2 0 6 » d = U'\b d = 8 -3 3 » x = U\d x = 1 -2 Forward substitution Back substitution