Math for CSLecture 31 LU Decomposition Pivoting Diagonalization Gram-Shcmidt Orthogonalization Lecture 3

Math for CSLecture 32 LU Decomposition A=LU Ax=b  LUx=b Define Ux=y Ly=bSolve y by forward substitution ERO’s must be performed on b as well as A The information about the ERO’s are stored in L Indeed y is obtained by applying ERO’s to b vector Ux=ySolve x by backward substitution

Math for CSLecture 33 LU Decomposition by Gaussian elimination Compact storage: The diagonal entries of L matrix are all 1’s, they don’t need to be stored. LU is stored in a single matrix. U=Gaussian eliminated matrix L=Multipliers used for elimination

Math for CSLecture 34 Pivoting Computer uses finite-precision arithmetic A small error is introduced in each arithmetic operation, error propagates When the pivotal element is very small, the multipliers will be large. Adding numbers of widely differening magnitude can lead to loss of significance. To reduce error, row interchanges are made to maximise the magnitude of the pivotal element

Math for CSLecture 35 Example: Without Pivoting 4-digit arithmetic Loss of significance

Math for CSLecture 36 Example: With Pivoting

Math for CSLecture 37 Pivoting procedures Eliminated part Pivotal column Pivotal row

Math for CSLecture 38 Row pivoting Most commonly used partial pivoting procedure Search the pivotal column Find the largest element in magnitude Then switch this row with the pivotal row

Math for CSLecture 39 Row pivoting Interchange these rows Largest in magnitude

Math for CSLecture 310 Column pivoting Interchange these columns Largest in magnitude

Math for CSLecture 311 Complete pivoting Interchange these columns Largest in magnitude Interchange these rows

Math for CSLecture 312 Row Pivoting in LU Decomposition When two rows of A are interchanged, those rows of b should also be interchanged. Use a pivot vector. Initial pivot vector is integers from 1 to n. When two rows (i and j) of A are interchanged, apply that to pivot vector.

Math for CSLecture 313 Modifying the b vector When LU decomposition of A is done, the pivot vector tells the order of rows after interchanges Before applying forward substitution to solve Ly=b, modify the order of b vector according to the entries of pivot vector

Math for CSLecture 314 Gauss-Jordan Diagonalization The elements above the diagonal are made zero at the same time that zeros are created below the diagonal

Math for CSLecture 315 Gauss-Jordan Diagonalization

Math for CSLecture 316 Gauss-Jordan Diagonalization

Math for CSLecture 317 Gram-Schmidt procedure for vector orthogonalization The purpose we can construct a set of orthonormal vectors u i from a set of n-dimensional vectors v i. 1≤i≤m And v i can be represented by the linear combination of u i

Math for CSLecture 318 G-S procedure (cont.) 1.Select a vector from v i arbitrarily, say v 1 2.By normalizing its length, we obtain the first vector, say 3.Select v 2 and subtract the projection of v 2 onto u 1, we get w 2 =v 2 – (v 2 u 1 )u 1

Math for CSLecture 319 G-S procedure (cont.) 4.And normalizing 5.Now we can check, that

Math for CSLecture 320 G-S procedure (cont.) 6.The procedure continues by selecting v 3 and subtract its projection on u 1 and u 2, we have w 3 =v 3 – (v 3 u 1 )u 1 – (v 3 u 2 )u 2 7.Then, the orthonormal vector u 3 is 8. By continuing this procedure, we shall construct the set of orthonormal vectors u i

Math for CSLecture 321 G-S procedure: Example 1/3 Consider v 1 =(0,1,1); v 2 =(1,-1,0); v 3 =(1,0,1) They are not orthogonal: (v i,v j )≠0. Following G-S:

Math for CSLecture 322 Example 2/3 We can check, that:

Math for CSLecture 323 Example 3/3 Why?