Conjugate Gradient Method invented by Hestenes and Stiefel around 1951 Conjugate Gradient Method It is an iterative method to solve the linear system of equations

Conjugate Gradient Method Example: Solve: 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 k=1 k=2 k=3 k=4 x1 x2 x3 X4 0 0.4716 0.9964 1.0015 1.0000 0 1.9651 1.9766 1.9833 2.0000 0 -0.8646 -0.9098 -1.0099 -1.0000 0 1.1791 1.0976 1.0197 1.0000 31.7 5.1503 1.0433 0.1929 0.0000

Quadratic function Define: Example: We want to solve the following linear system Define: quadratic function Example:

Quadratic Function Example: Remark:

Minimization equivalent ot linear system Remark: Problem (1) Problem (2) IDEA: Search for the minimum

Conjugate Gradient Method Example: minimum

Minimum IDEA: Search for the minimum Remark:

Conjugate Gradient Method “search direction” “step length”

Conjugate Gradient Method vectors constants

Conjugate Gradient Method

Conjugate Gradient Method

INNER PRODUCT

Inner Product DEF: Example: Example: We say that Is an inner product if Example: Example: A is SPD We define the norm

Inner Product DEF: DEF: where A is SPD Example:

Conjugate Gradient

Conjugate Gradient Method

Conjugate Gradient Method

Conjugate Gradient Method HW:

Conjugate Gradient Method Lemma:[Elman,Silvester,Wathen Book] vectors Orthogonal A-Orthogonal

Error and Residual vectors REMARK REMARK Orthogonal A-Orthogonal Minimizes the A-norm of the error

Conjugate Gradient Method 0.0000 0.4716 0.9964 1.0015 1.0000 0.0000 1.9651 1.9766 1.9833 2.0000 0.0000 -0.8646 -0.9098 -1.0099 -1.0000 0.0000 1.1791 1.0976 1.0197 1.0000 6.0000 4.9781 -0.1681 -0.0123 0.0000 25.0000 -0.5464 0.0516 0.1166 -0.0000 -11.0000 -0.1526 -0.8202 0.0985 0.0000 15.0000 -1.1925 -0.6203 -0.1172 -0.0000 6.0000 5.1362 0.0427 -0.0108 25.0000 0.1121 0.0562 0.1185 -11.0000 -0.4424 -0.8384 0.0698 15.0000 -0.7974 -0.6530 -0.1395 0.0786 0.1022 0.1193 0.1411 0.0713 0.0263 0.0410 0.0342

Connection to Lanczos

Introduction to Krylov Subspace Methods DEF: Krylov sequence Example: Krylov sequence 1 11 118 1239 12717 1 12 141 1651 19446 1 10 100 989 9546 1 10 106 1171 13332 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8

Introduction to Krylov Subspace Methods DEF: Krylov subspace Example: Krylov subspace 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 DEF: Example: Krylov matrix

Introduction to Krylov Subspace Methods DEF: Example: Krylov matrix Remark:

Lanczos method Lanczos: The Lanczos algorithm is defined as follows An orthogonal basis for