1. Roots of Equations
Open Methods
Second Term 05/06

2. The following root finding methods will be introduced:
A. Bracketing Methods
A.1. Bisection Method
A.2. Regula Falsi
B. Open Methods
B.1. Fixed Point Iteration
B.2. Newton Raphson's Method
B.3. Secant Method
Second Term 05/06

3. B. Open Methods
To find the root for f(x) = 0, we construct a magic formulae
xi+1 = g(xi)
to predict the root iteratively until x converge to a root. However, x may diverge!
Bisection method
Open method (diverge)
Open method (converge)
Second Term 05/06

4. What you should know about Open Methods
How to construct the magic formulae g(x)?
How can we ensure convergence?
What makes a method converges quickly or diverge?
How fast does a method converge?
Second Term 05/06

5. B.1. Fixed Point Iteration
Also known as one-point iteration or successive substitution
To find the root for f(x) = 0, we rearrange f(x) = 0 so that there is an x on one side of the equation.
If we can solve g(x) = x, we solve f(x) = 0.
We solve g(x) = x by computing
until xi+1 converges to xi.
Second Term 05/06

6. Fixed Point Iteration – Example
Reason: When x converges, i.e. xi+1  xi
Second Term 05/06

7. Example Find root of f(x) = e-x - x = 0. (Answer: α= 0.56714329) i xi
100.0 1
76.3 2
171.8
35.1 3
46.9
22.1 4
38.3
11.8 5
17.4
6.89 6
11.2
3.83 7
5.90
2.20 8
3.48
1.24 9
1.93
0.705 10
1.11
0.399
Second Term 05/06

8. Two Curve Graphical Method
The point, x, where the two curves,
f1(x) = x and
f2(x) = g(x),
intersect is the solution to f(x) = 0.
Demo
Second Term 05/06

9. There are infinite ways to construct g(x) from f(x).
Fixed Point Iteration
There are infinite ways to construct g(x) from f(x).
For example,
(ans: x = 3 or -1)
Case a:
Case b:
Case c:
So which one is better?
Second Term 05/06

10. Converge! Diverge! Converge, but slower x0 = 4 x1 = 3.31662
x0 = 4
x1 = 6.5
x1 =
x1 =
Converge!
Diverge!
Converge, but slower
Second Term 05/06

11. How to choose g(x)?
Can we know which function g(x) would converge to solution before we do the computation?
Second Term 05/06

12. Convergence of Fixed Point Iteration
By definition
Fixed point iteration
Second Term 05/06

13. Convergence of Fixed Point Iteration
According to the derivative mean-value theorem, if a g(x) and g'(x) are continuous over an interval xi ≤ x ≤ α, there exists a value x = ξ within the interval such that
Therefore, if |g'(x)| < 1, the error decreases with each iteration. If |g'(x)| > 1, the error increase.
If the derivative is positive, the iterative solution will be monotonic.
If the derivative is negative, the errors will oscillate.
Second Term 05/06

14. Demo (a) |g'(x)| < 1, g'(x) is +ve converge, monotonic
(b) |g'(x)| < 1, g'(x) is -ve
converge, oscillate
(c) |g'(x)| > 1, g'(x) is +ve
diverge, monotonic
(d) |g'(x)| > 1, g'(x) is -ve
diverge, oscillate
Demo
Second Term 05/06

15. Fixed Point Iteration Impl. (as C function)
// x0: Initial guess of the root
// es: Acceptable relative percentage error
// iter_max: Maximum number of iterations allowed
double FixedPt(double x0, double es, int iter_max) {
double xr = x0; // Estimated root
double xr_old; // Keep xr from previous iteration
int iter = 0; // Keep track of # of iterations
do {
xr_old = xr;
xr = g(xr_old); // g(x) has to be supplied
if (xr != 0)
ea = fabs((xr – xr_old) / xr) * 100;
iter++;
} while (ea > es && iter < iter_max);
return xr; }
Second Term 05/06

16. The following root finding methods will be introduced:
A. Bracketing Methods
A.1. Bisection Method
A.2. Regula Falsi
B. Open Methods
B.1. Fixed Point Iteration
B.2. Newton Raphson's Method
B.3. Secant Method
Second Term 05/06

17. B.2. Newton-Raphson Method
Use the slope of f(x) to predict the location of the root.
xi+1 is the point where the tangent at xi intersects x-axis.
Second Term 05/06

18. Newton-Raphson Method
What would happen when f '(α) = 0?
For example, f(x) = (x-1)2 = 0
Second Term 05/06

19. Error Analysis of Newton-Raphson Method
By definition
Newton-Raphson method
Second Term 05/06

20. Error Analysis of Newton-Raphson Method
Suppose α is the true value (i.e., f(α) = 0).
Using Taylor's series
When xi and α are very close to each other, ξ is between xi and α.
The iterative process is said to be of second order.
Second Term 05/06

21. The Order of Iterative Process (Definition)
Using an iterative process we get xk+1 from xk and other info.
We have x0, x1, x2, …, xk+1 as the estimation for the root α.
Let δk = α – xk
Then we may observe
The process in such a case is said to be of p-th order.
It is called Superlinear if p > 1.
It is called Linear if p = 1.
It is called Sublinear if p < 1.
Second Term 05/06

22. Error of the Newton-Raphson Method
Each error is approximately proportional to the square of the previous error. This means that the number of correct decimal places roughly doubles with each approximation.
Example: Find the root of f(x) = e-x - x = 0
(Ans: α= )
Error Analysis
Second Term 05/06

23. Error Analysis i xi εt (%) |δi| estimated |δi+1| 100 0.56714329 0.0582
100
0.0582 1
11.8 2
0.147 3
2.83x10-15 4
< 10-8
Second Term 05/06

24. Newton-Raphson vs. Fixed Point Iteration
Find root of f(x) = e-x - x = 0.
(Answer: α= )
Fixed Point Iteration with i xi
εa (%)
εt (%)
100.0 1
76.3 2
171.8
35.1 3
46.9
22.1 4
38.3
11.8 5
17.4
6.89 6
11.2
3.83 7
5.90
2.20 8
3.48
1.24 9
1.93
0.705 10
1.11
0.399
Newton-Raphson i xi
εt (%)
|δi|
100 1
11.8 2
0.147 3 4
< 10-8
Second Term 05/06

25. Pitfalls of the Newton-Raphson Method
Sometimes slow
iteration x
0.5 1
51.65 2
46.485 3 4 5 …
Infinity
Second Term 05/06

26. Pitfalls of the Newton-Raphson Method
Figure (a)
An infection point (f"(x)=0) at the vicinity of a root causes divergence.
Figure (b)
A local maximum or minimum causes oscillations.
Second Term 05/06

27. Pitfalls of the Newton-Raphson Method
Figure (c)
It may jump from one location close to one root to a location that is several roots away.
Figure (d)
A zero slope causes division by zero.
Second Term 05/06

28. Overcoming the Pitfalls?
No general convergence criteria for Newton-Raphson method.
Convergence depends on function nature and accuracy of initial guess.
A guess that's close to true root is always a better choice
Good knowledge of the functions or graphical analysis can help you make good guesses
Good software should recognize slow convergence or divergence.
At the end of computation, the final root estimate should always be substituted into the original function to verify the solution.
Second Term 05/06

29. The following root finding methods will be introduced:
A. Bracketing Methods
A.1. Bisection Method
A.2. Regula Falsi
B. Open Methods
B.1. Fixed Point Iteration
B.2. Newton Raphson's Method
B.3. Secant Method
Second Term 05/06

30. B.2. Secant Method
Newton-Raphson method needs to compute the derivatives.
The secant method approximate the derivatives by finite divided difference.
From Newton-Raphson
method
Second Term 05/06

31. Secant Method
Second Term 05/06

32. Secant Method – Example
Find root of f(x) = e-x - x = 0 with initial estimate of
x-1 = 0 and x0 = 1.0. (Answer: α= ) i
xi-1 xi
f(xi-1)
f(xi)
xi+1 εt 1
8.0 %
0.58 % 2 %
Again, compare this results obtained by the Newton-Raphson method and simple fixed point iteration method.
Second Term 05/06

33. Comparison of the Secant and False-position method
Both methods use the same expression to compute xr.
They have different methods for the replacement of the initial values by the new estimate. (see next page)
Second Term 05/06

34. Comparison of the Secant and False-position method
Second Term 05/06

35. Comparison of the Secant and False-position method
Second Term 05/06

36. Modified Secant Method
Needs only one instead of two initial guess points
Replace xi-1 - xi by δxi and approximate f'(x) as
From Newton-Raphson method,
Second Term 05/06

37. Modified Secant Method
Find root of f(x) = e-x - x = 0 with initial estimate of
x0 = 1.0 and δ=0.01. (Answer: α= ) i xi
xi+δxi
f(xi)
f(xi+δxi)
xi+1 1
1.01 2
Compared with the Secant method i
xi-1 xi
f(xi-1)
f(xi)
xi+1 εt 1
8.0 %
0.58 % 2 %
Second Term 05/06

38. Modified Secant Method – About δ
If δ is too small, the method can be swamped by round-off error caused by subtractive cancellation in the denominator of
If δ is too big, this technique can become inefficient and even divergent.
If δ is selected properly, this method provides a good alternative for cases when developing two initial guess is inconvenient.
Second Term 05/06

39. The following root finding methods will be introduced:
A. Bracketing Methods
A.1. Bisection Method
A.2. Regula Falsi
B. Open Methods
B.1. Fixed Point Iteration
B.2. Newton Raphson's Method
B.3. Secant Method
Can they handle multiple roots?
Second Term 05/06

40. Multiple Roots
A multiple root corresponds to a point where a function is tangent to the x axis.
For example, this function has a double root.
f(x) = (x – 3)(x – 1)(x – 1)
= x3 – 5x2 + 7x - 3
For example, this function has a triple root.
f(x) = (x – 3)(x – 1)(x – 1) (x – 1)
= x4 – 6x3 +12x2 - 10x + 3
Second Term 05/06

41. Odd multiple roots cross the axis. (Figure (b))
Even multiple roots do not cross the axis. (Figure (a) and (c))
Second Term 05/06

42. Difficulties when we have multiple roots
Bracketing methods do not work for even multiple roots.
f(α) = f'(α) = 0, so both f(xi) and f'(xi) approach zero near the root. This could result in division by zero. A zero check for f(x) should be incorporated so that the computation stops before f'(x) reaches zero.
For multiple roots, Newton-Raphson and Secant methods converge linearly, rather than quadratic convergence.
Second Term 05/06

43. Modified Newton-Raphson Methods for Multiple Roots
Suggested Solution 1:
Disadvantage:
work only when m is known.
Second Term 05/06

44. Modified Newton-Raphson Methods for Multiple Roots
Suggested Solution 2:
Second Term 05/06

45. Example of the Modified Newton-Raphson Method for Multiple Roots
Original Newton Raphson method i xi
εt (%)
100 1 57 2 31 3 17 4
8.7 5
4.4 6
2.2
The method is linearly convergent toward the true value of 1.0.
Second Term 05/06

46. Example of the Modified Newton-Raphson Method for Multiple Roots
For the modified algorithm i xi
εt (%)
100 1 11 2
0.31 3
Second Term 05/06

47. Example of the Modified Newton-Raphson Method for Multiple Roots
How about their performance on finding the single root? i
Standard
εt (%)
Modified 4 33 1
3.4 13 12 2
3.1
3.3
6.0 3
0.29
1.3
0.0025
0.05 5
2x10-7
7.7x10-5
Second Term 05/06

48. Modified Newton-Raphson Methods for Multiple Roots
What's the disadvantage of the modified Newton-Raphson Methods for multiple roots over the original Newton-Raphson method?
Note that the Secant method can also be modified in a similar fashion for multiple roots.
Second Term 05/06

49. Summary of Open Methods
Unlike bracketing methods, open methods do not always converge.
Open methods, if converge, usually converge more quickly than bracketing methods.
Open methods can locate even multiple roots whereas bracketing methods cannot. (why?)
Second Term 05/06

50. Study Objectives Understand the graphical interpretation of a root
Understand the differences between bracketing methods and open methods for root location
Understand the concept of convergence and divergence
Know why bracketing methods always converge, whereas open methods may sometimes diverge
Realize that convergence of open methods is more likely if the initial guess is close to the true root.
Second Term 05/06

51. Study Objectives
Understand what conditions make a method converge quickly or diverge
Understand the concepts of linear and quadratic convergence and their implications for the efficiencies of the fixed-point-iteration and Newton-Raphson methods
Know the fundamental difference between the false-position and secant methods and how it relates to convergence
Understand the problems posed by multiple roots and the modifications available to mitigate them
Second Term 05/06

52. Analysis of Convergent Rate
Suppose g(x) converges to the solutionα, then the Taylor series of g(α) about xi can be expressed as
By definition
Thus (1) becomes
Second Term 05/06

53. Analysis of Convergent Rate
When xi is very close to the solution, we can rewrite (2) as
Suppose g(n) exists and the nth term is the first non-zero term, then
Thus to analyze the convergent rate, we can find the smallest n such that g(n)(α) ≠ 0.
Second Term 05/06