Iterative Methods for Solving Linear Systems Leo Magallon & Morgan Ulloa

What is an Iterative Method - An alternative method for solving the linear systems problem Ax=b where A is an nxn matrix with n equations and n unknowns. - These methods use a repetitive algorithm to generate sequences of vectors that steadily approach your solution. We will discuss two types of iterative methods:  Jacobi  Gauss-Seidel

Convergence Convergence is when the outputs of your functions stop changing significantly. In other words they converge towards a solution. Iterative methods rely on the functions’ ability to converge towards a solution, if they don’t converge then you cannot find an answer. We can say that a system will converge if it is diagonally dominant. When the absolute value of each diagonal entry is greater than the absolute value of the sum of every component in its row.

The linear systems problem Ax=b Let’s look at the following system From what we’ve learned in class we know how to solve this system by taking rref(A) in an augmented matrix. So lets solve our system using elementary row operations.

As we can see by taking rref(A) the solution to our system is x 1 = 1 x 2 = 2 Lets check our answer: 7(1) – 1(2)= 5 3(1) – 5(2)= -7 ☺ Correct!

So why use iterative methods?

When using elementary row operations by hand it is likely that calculating errors will be made (especially when the matrix is full of fractions) Iterative methods can be quicker if the matrix is simple or has many zero components. You can stop calculating when your answers start to converge, whereas with row operations you must work all the way to rref(A) Round off errors may actually accelerate convergence because you are jumping more quickly towards your solution.

Let’s look at Jacobi’s method From our 2x2 matrix we can see that there are 2 equations with 2 unknowns, and the matrix is diagonally dominant; therefore it will converge. 7x 1 – 1x 2 = 5 3x 1 – 5x 2 = -7 Step 1: solve the first equation for x 1 and solve the second equation for x 2 x 1 = 5 + x 2 x 2 = 7 + 3x Step 2: choose an initial approximation. We choose x 1 = 0x 2 = 0

Step 3: plug in your chosen values and solve for x 1 and x 2. X 1 = 5 + 1(0) = 5/7 X 2 = 7 + 3(0) = 7/5 7 5 Step 4: use new values for x 1 and x 2 and input them into your original functions…. continue this process until it converges to your solution. Here we can see the solution is converging to x 1 =1 x 2 = 2 at around 6 iterations. the same as our answer from rref(A).

Gauss-Seidel Almost identical to Jacobi’s method except it converges faster because we use our new outputs as soon as we can. Let’s see how it works… Step1: from our two equations, solve for x 1 and x 2 7x 1 – 1x 2 = 5 3x 1 – 5x 2 = -7 x 1 = 5 + x 2 x 2 = 7 + 3x 1 7 5

Step 2: choose an initial approximation. we choose Step 3: plug in your initial value and solve for x 1 x 1 = 5 + 1(0) = 5/7 7 Step 4: now use this new value for x 1 to solve for x 2 x 2 = 7 + 3( 5/7 ) = 64/35 ≈ solve again for x 1 x = 5 + 1( 64/35 )≈ x 1 = 0x 2 = 0

Step 5: continue to input your newest values for x 1 and x 2 into the functions until they converge towards a solution. Here we can see that the solution is converging to x 1 = 1 x 2 = 2 at around 4 iterations.

iteration shown graphically David Strong's applet for solving Ax=b using iterative methodsDavid Strong's applet for solving Ax=b using iterative methods

Iterative methods rock!