Presentation is loading. Please wait.

Presentation is loading. Please wait.

Chapter 13 Gaussian Elimination (III) Bunch-Parlett diagonal pivoting Reference: James R. Bunch and Linda Kaufman, Some Stable Methods for Calculating.

Similar presentations


Presentation on theme: "Chapter 13 Gaussian Elimination (III) Bunch-Parlett diagonal pivoting Reference: James R. Bunch and Linda Kaufman, Some Stable Methods for Calculating."— Presentation transcript:

1 Chapter 13 Gaussian Elimination (III) Bunch-Parlett diagonal pivoting Reference: James R. Bunch and Linda Kaufman, Some Stable Methods for Calculating Inertia and Solving Symmetric Linear Systems, Mathematics of Computation, volume 31, number 137, January 1977, page Speaker: Lung-Sheng Chien

2 OutLine Preliminary - 1x1, 2x2 pivoting in Gaussian Elimination - real symmetric indefinite matrix Symmetric permutation LDL’ decomposition (diagonal pivoting) Example of complete diagonal pivoting Algorithm of complete diagonal pivoting

3 It is easy to show from Gaussian Elimination Pivoting in Traditional Gaussian Elimination consider where and then one step LU-factorization leads to We use principal submatrix as pivoting, calledpivoting since Question: can we use or higherpivoting, i.e.

4 then one-step GE leads to If principal sub-matrixis non-singular, Example 1: pivoting 2x2 pivoting in block Gaussian Elimination where and

5 Example 2: pivoting where and 3x3 pivoting in block Gaussian Elimination Question: what is criterion to choose 1x1, 2x2 or 3x3 pivoting? stabilityperformance Question: what is the cost to estimate the criterion ?

6 Real symmetry indefinite matrix A is symmetric impliesand From linear algebra, if A is real symmetric, then A has real spectrum decomposition ( 譜分解 ) and or write in matrix formand without loss of generality, we can rearrange eigen-values to be increasing Definition: suppose matrix A is real symmetric, then we say “A is indefinite” if there exists an eigen-value of A less than zero. If A is nonsingular, then Thereafter we focus on LU-decomposition of symmetry indefinite matrix A

7 LU-factorization for real symmetry indefinite matrix A factorization where and Question: why not LU-decomposition? A is real symmetric, we only store lower triangle part of A, say

8 row-major based Pointer array to store lower triangular part of A col-major based Question: can you find another representation for lower part of matrix A ? Question: what is the cost to fetch one element of A ? That is, operation count for Question: if one uses row-major to store lower part of matrix A, then how to fetch a column of A efficiently?

9 OutLine Preliminary Symmetric permutation - change diagonal element to (1,1) position - change off-diagonal element to (2,1) position - implementation of symmetric permutation LDL’ decomposition (diagonal pivoting) Example of complete diagonal pivoting Algorithm of complete diagonal pivoting

10 Define permutation matrix How to do permutation on real symmetry indefinite matrix A interchange row 2, 3 interchange column 2, 3 Let interchange row 2, interchange column 2, Symmetric permutation:

11 Change diagonal element to (1,1) position [1] suppose we want to changeto position (1,1), then consider permutation and do symmetric permutation on A, say interchange col 1, 3 interchange row 1, 3 Question: we only store lower triangle part of matrix A, above permutation does not work, how to modify it? Observation 1:

12 Observation 2: we only need to update lower triangle part of A (diagonal term is excluded ) diagonal term does not changed we have changed does not changed Change diagonal element to (1,1) position [2]

13 interchange col 1, interchange row 1, Keep lower triangle part Fill-in diagonal term Fill-in A(3,1) Change diagonal element to (1,1) position [3] Question: any compact form to represent above interchange?

14 Change diagonal element to (1,1) position [4] case 1: case 2: 2 6

15 interchanges rows or columns Change diagonal element to (1,1) position [5] Step 1: interchange position (1,1) and (k,k), i.e. For general real symmetric matrix A with dimension n, suppose we want to change keep position (k,1) and (1,k), i.e. does not changed

16 Change diagonal element to (1,1) position [6] Step 2: interchange column 1 and k below row k+1, i.e Therefore

17 Change diagonal element to (1,1) position [7] Step 3: interchange column 1 (from row-2 to row-(k-1)) and row k (from col-2 to col-(k-1)) Therefore

18 Implementation of symmetric permutation: swapping overhead [1] Suppose we use col-major based pointer array to store real symmetric matrix A Step 2: interchanging column 1 and k is O.K since memory block is contiguous. Contiguous memory block

19 Implementation of symmetric permutation: swapping overhead [2] Step 3: interchanging column 1 and row k, i.e. is NOT efficient sinceis NOT contiguous. NOT contiguous, swapping is very slow. Observation: the situation is the same even we choose row-major based pointer array as storage strategy. Solution: in order to avoid high swapping overhead, we should avoid permutation.

20 Change off-diagonal element to (2,1) position [1] suppose we want to changeto position (2,1), then consider first permutation which change first index from 4 to interchange col 2, 4 interchange row 2, 4 and second permutationwhich change second index from 3 to interchange col 1, interchange row 1,

21 interchange row and col 2, interchange row and col 1, Change off-diagonal element to (2,1) position [2] Observation: two permutation matrices can be computed easily since 2  4 and 1  3 are independent.

22 Change off-diagonal element to (2,1) position [3] HOW to transformto position (2,1), under only lower triangle part of A is stored? interchange row/col 2, 4 Observation 1:anddoes not changed Step 1: interchange position (4,4) and (2,2),

23 Change off-diagonal element to (2,1) position [4] Observation 2: we only need to update lower triangle part of A (diagonal term is excluded ) Step 2: interchange row 2 and 4 below col 2, i.e interchange row 2, interchange col 2, NOTE: Interchanging column 2, 4 does not change position (2,1) and (4,1), it suffices to interchange position (2,1) and (4,1) first.

24 Change off-diagonal element to (2,1) position [5] Step 3: interchange column 2 (from row-3 to row-3) and row 4 (from col-3 to col-3)

25 interchanges rows or columns Change off-diagonal element to (2,1) position [6] Step 1: interchange position (2,2) and (k,k), i.e. For general real symmetric matrix A with dimension n, suppose we want to change keep position (k,2) and (2,k), i.e. does not changed

26 Change off-diagonal element to (2,1) position [7] Step 2: interchange column 2 and k below row k+1, i.e Therefore

27 Change off-diagonal element to (2,1) position [8] Step 3: interchange row 2 (column 1) and row k (column1), i.e Therefore

28 Change off-diagonal element to (2,1) position [9] Step 4: interchange column 2 (from row-3 to row-(k-1)) and row k (from col-3 to col-(k-1)) Therefore

29 OutLine Preliminary Symmetric permutation LDL’ decomposition (diagonal pivoting) - growth rate of 1x1, 2x2 pivoting - criterion for pivot strategy - comparison to complete pivoting in GE Example of complete diagonal pivoting Algorithm of complete diagonal pivoting

30 LDL’ decomposition: 1x1, 2x2 pivoting diagonal pivoting method with complete pivoting: Bunch-Parlett, “Direct methods fro solving symmetric indefinite systems of linear equations,” SIAM J. Numer. Anal., v. 8, 1971, pp diagonal pivoting method with partial pivoting: Bunch-Kaufman, “Some Stable Methods for Calculating Inertia and Solving Symmetric Linear Systems,” Mathematics of Computation, volume 31, number 137, January 1977, page

31 maximal element of matrix A maximal diagonal element of matrix A Case 1: s = 1 Growth rate in LDL’ decomposition [1] is of order Define and constant with and where

32 Growth rate in LDL’ decomposition [2] Therefore Observation: in PA=LU, we choose such that implies Therefore Observation: large element growth will not occur for apivot if is large relative to

33 Growth rate in LDL’ decomposition [3] Case 2: s = 2 where with and with and implies and

34 Growth rate in LDL’ decomposition [4] for Therefore

35 Growth rate in LDL’ decomposition [5] Therefore

36 Fix a number( later on, we will determine optimal value of ) Case 1: maximal element of matrix A maximal diagonal element of matrix A supposewe interchange 1st and k-th rows and columns by introducing permutation matrixand do symmetric permutation Then doonwithpivot, written as Criterion for pivot strategy [1]

37 Criterion for pivot strategy [2] Recall forpivot,, we have growth rate and Now for and Case 2: suppose, we interchange q-th and 1st rows and columns, and then r-th and 2nd rows and columns by introducing permutation matrix and do symmetric permutation Question: we must change q  1 first, then change r  2, why?

38 Criterion for pivot strategy [3] but (boundary case) transforms

39 Criterion for pivot strategy [4] Then doonwithpivot, written as pivot Recall forpivot,, we have growth rate and

40 Criterion for pivot strategy [5] Now for To sum up Case 1: and Case 2: and Question: How to choose value of

41 Criterion for pivot strategy [6] worst case analysis : choosesuch that growth rate ofpivot +pivotgrowth rate ofpivot or equivalentlygrowth rate ofpivotgrowth rate ofpivot Define wheresatisfies Exercise: verify and

42 Diagonal pivoting versus complete pivoting of Gaussian Elimination [1] We must search for the largest element in each reduced matrix, this is a complete pivoting strategy analogous to Gaussian Elimination with complete pivoting maximal element of matrix A maximal diagonal element of matrix A 1 (1) find a such that (2) swap rowand row thenwith (2) swap columnand column complete pivoting of GE (Gaussian Elimination)

43 growth rate 2 diagonal pivoting complete pivoting in GE Diagonal pivoting versus complete pivoting of Gaussian Elimination [2] Remark: Bunch [1] proves that the element growth in the diagonal pivoting method in comparison with with complete pivoting is bounded by for Gaussian Elimination with complete pivoting, where [1] J.R. Bunch, “Analysis of the diagonal pivoting method”, SIAM J. Numer. Anal., v. 8, 1971, pp

44 OutLine Preliminary Symmetric permutation LDL’ decomposition (diagonal pivoting) Example of complete diagonal pivoting Algorithm of complete diagonal pivoting

45 swap row/column we choosepivot andby permutation matrix Example (complete pivoting) [1]

46 Example (complete pivoting) [2] we choosepivot swap row/columnby permutation matrix Recursively do the same procedure for such that

47 Example (complete pivoting) [3] Do symmetry permutation forand we dopivot on

48 Example (complete pivoting) [4] Or write in original matrix form

49 Example (complete pivoting) [5] In practice, we have two key issues we only store lower part of matrix col-major based

50 Example (complete pivoting) [6] we store decomposition 2 andinto original Question: How can you judge correct decomposition from

51 Example (complete pivoting) [7] Case 1: four 1x1 pivot Case 2: two 2x2 pivot

52 Example (complete pivoting) [8] Case 3: 1x1 pivot + 2x2 pivot + 1x1 pivot Case 4: 2x2 pivot + 1x1 pivot + 1x1 pivot Solution: we need an array to record pivot sequence pivot x2 pivot 1x1 pivot

53 OutLine Preliminary Symmetric permutation LDL’ decomposition (diagonal pivoting) Example of complete diagonal pivoting Algorithm of complete diagonal pivoting

54 we have compute update original matrix combines all lower triangle matrix and store in Algorithm ( PAP’ = LDL’ ) [1] Given symmetric indefinite matrix, construct initial lower triangle matrix while use permutation vector to record permutation matrix let, and, where

55 compute and 1 Algorithm ( PAP’ = LDL’ ) [2] Case 1: define permutationto do symmetric permutation To compute, we only update lower triangle of 3 then 2

56 Algorithm ( PAP’ = LDL’ ) [3] do 1x1 pivot : To compute We only update lower triangle matrix 4 then 5 6 and

57 Algorithm ( PAP’ = LDL’ ) [4] Case 2: define permutation to interchange q-th and k-th rows and columns,and then r-th and (k+1)-th rows and columns 7 and then To compute, we only update lower triangle of (1) do interchange row/col then

58 Algorithm ( PAP’ = LDL’ ) [5] (2) do interchange row/col then where 3

59 Algorithm ( PAP’ = LDL’ ) [6] To compute 10 then (1) do interchange row 11 (2) do interchange row

60 Algorithm ( PAP’ = LDL’ ) [7] do 2x2 pivot : 12 then 13 and means is 2x2 pivot endwhile

61 Question: recursion implementation Initialization - check algorithm holds for k=1 Recursion formula - check algorithm holds for k or k+1, if k-1 is true Termination condition - check algorithm holds for k=n-1 Breakdown of algorithm - check which condition PAP’=LDL’ fails No extension: algorithm works only for square, symmetric indefinite matrix. normal abnormal Extension of algorithm

62 MATLAB implementation [1] Given a (full) symmetric indefinite matrix, compute factorization return four quantities Example : return : Remark: try to neglect upper triangle part ofin MATLAB implementation

63 MATLAB implementation [2] When factorization is complete,, you need to provide linear solver define, then by forward substitution define, then by diagonal block inversion Example:, then define, thenby backward substitution However you cannot transposeexplicitly in MATLAB , you must scanonce to determine


Download ppt "Chapter 13 Gaussian Elimination (III) Bunch-Parlett diagonal pivoting Reference: James R. Bunch and Linda Kaufman, Some Stable Methods for Calculating."

Similar presentations


Ads by Google