Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm Harder. All rights reserved. ECE 204 Numerical Methods for Computer Engineers

Fixed-Point Iteration In this topic, we will look at: –how fixed-point iteration can be used to solve certain problems numerically, and –some of the various limitations of iterative numerical methods including: determining convergence, slow convergence, and non-convergence

Fixed-Point Iteration Everyone has at some point punched in an arbitrary number on a scientific calculator and started pushing one of the buttons We will look at two examples: sin and cos

Fixed-Point Iteration Starting with x = 0, if I repeatedly hit the cos button, I get the sequence: x = 0 x = 1 x = x = x = x = x = x = x = x = x = x = x = x = x = x = x = x = x =

Fixed-Point Iteration Looking at the first three digits of the last three iterations, we note that they appear to be converging:

Fixed-Point Iteration After applying cos 100 times using Matlab, I get the sequence: x = x = x = x = x = x = x = x = x = x = x =

Fixed-Point Iteration If, however, you set your calculator to degrees instead of radians, you would have found a different sequence: >> x = 0; >> for i = 1:6 x = cos(x/180*pi) end x = 1 x = x = x = x =

Fixed-Point Iteration In this case, the sequence converged significantly faster, but to what?

Fixed-Point Iteration Given an equation of the form x = f(x) if |f (1) (x)| < 1 in a neighbourhood of a solution to this equation, then for any initial point x 0 in this neighbourhood, the sequence x k + 1 = f(x k ) will converge to the solution to this equation

Fixed-Point Iteration The examples we have been looking at were finding points such that: x = cos(x) This will differ depending on whether you are using radians or degrees:

Fixed-Point Iteration We will now look at one of the longest known examples of fixed-point iteration Starting with any value and repeated apply the function f(x) = x/2 + 1/x For centuries, it has been known that this converges to

Fixed-Point Iteration To see why, recall that this converges to a solution of x = x/2 + 1/x which is also a solution of x 2 = x 2 /2 + 1 or x 2 /2 = 1 x 2 = 2

Newton’s Method We also get this sequence by applying Newton’s method which was covered in first year The function f(x) = x 2 – 2 has two roots, and we can find these roots using Newton’s method:

Newton’s Method Thus, we iterate Looking at the right hand side, it simplifies to:

Fixed-Point Iteration For example, consider the equation x = 2x(x – 1) This has two solutions, namely x = 0 and x = 0.5 If we start with x 0 = 0.1 and define f(x) = 2x(x – 1), we get the sequence

Fixed-Point Iteration Sequence is kxkxk |x k – x k – 1 | 00.1–

Fixed-Point Iteration To solve any such problem, we could continue to iterate until the sequence no longer changes, however, this does not always work Consider the slight modification, solving x = 2.4x(x – 1) which has the solution x = 7/12 ≈ ⋅⋅

Fixed-Point Iteration Starting with x 0 = 0.1, we converge much more slowly, and after 32 iterations: kxkxk |x k – x k – 1 | – – – – – – – – –11

Fixed-Point Iteration This sequence will never converge, and therefore we need a weaker halting condition We will continue iterating until: |x k – x k – 1 | <  step where  step is a suitably chosen predetermined criteria

Fixed-Point Iteration Unfortunately, this may also lead to erroneous results

Fixed-Point Iteration Suppose we are trying to solve x = sin(x) which has only one solution x = 0 After 1000 iterations, we still have: x 1000 = x 1001 = x 1002 = x 1003 =

Fixed-Point Iteration In this case, the rate of convergence is very, very slow The following table lists the number of iterations required to satisfy the terminating condition:

Fixed-Point Iteration The following table shows how many iterations are required  step Iterations xkxk 10 – – – – – – – –

Fixed-Point Iteration Notice that a small step size does not suggest that we are anywhere near the solution? After 20 million iterations: –the step size is less than 10 –12, but –the value ·· has only three zeros after the decimal place

Fixed-Point Iteration Therefore, we must restrict the number of iterations required to some fixed amount, say N Thus, if we iterate N times and have not satisfied our halting conditions, we stop and indicate that the method failed to converge

Fixed-Point Iteration If we stop due to iterating N times, this may mean that either: –a solution does not exist, –we chose a poor initial condition, or –we are using an inappropriate method for finding a solution

Fixed-Point Iteration Alternatively, to find a solution to x = sin(x), we could define f(x) = sin(x) – x and then iterate Thus, we get: x 0 = 0.1 x 1 = – x 2 =

Fixed-Point Iteration In this topic, we have seen a number of examples of fixed-point iteration –it may be possible to solve equations of the form x = f(x) by repeated applying f(x) –the form of the iteration will affect the convergence –it is necessary to define a halting condition stopping if the difference is less than  step –the sequence may not converge so we may have to halt after some fixed number of iterations

Usage Notes These slides are made publicly available on the web for anyone to use If you choose to use them, or a part thereof, for a course at another institution, I ask only three things: –that you inform me that you are using the slides, –that you acknowledge my work, and –that you alert me of any mistakes which I made or changes which you make, and allow me the option of incorporating such changes (with an acknowledgment) in my set of slides Sincerely, Douglas Wilhelm Harder, MMath