1. Lecture 5 Newton-Raphson Method
Assumptions
Interpretation
Examples
Convergence Analysis
Reading Assignment: Sections 6.2
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

2. Root finding Problems
Many problems in Science and Engineering are expressed as
These problems are called root finding problems
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

3. Solution Methods
Several ways to solve nonlinear equations are possible.
Analytical Solutions
possible for special equations only
Graphical Solutions
Useful for providing initial guesses for other methods
Numerical Solutions
Open methods
Bracketing methods
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

4. Bracketing Methods
In bracketing methods, the method starts with an interval that contains the root and a procedure is used to obtain a smaller interval containing the root.
Examples of bracketing methods :
Bisection method
False position method
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

5. Open Methods
In the open methods, the method starts with one or more initial guess points. In each iteration a new guess of the root is obtained.
Open methods are usually more efficient than bracketing methods
They may not converge to the root.
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

6. These will be covered in EE3561
Solution Methods
Many methods are available to solve nonlinear equations
Bisection Method
Newton’s Method
Secant Method
False position Method
Muller’s Method
Bairstow’s Method
Fixed point iterations
……….
These will be covered in EE3561
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

7. Newton-Raphson Method (also known as Newton’s Method)
Given an initial guess of the root x0 , Newton-Raphson method uses information about the function and its derivative at that point to find a better guess of the root.
Assumptions:
f (x) is continuous and first derivative is known
An initial guess x0 such that f ’(x0) ≠0 is given
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

8. EE3561_Unit 2
(c)AL-DHAIFALLAH1435

9. EE3561_Unit 2
(c)AL-DHAIFALLAH1435

10. Newton’s Method
F.m
FP.m
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

11. Example
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

12. Example Iteration xk f(xk) f’(xk) xk+1 |xk+1 –xk| 4 33 3 1 9 16 2.43754 33 3 1 9 16
2.4375
0.5625 2
2.0369
9.0742
2.2130
0.2245
0.2564
6.8404
2.1756
0.0384
0.0065
6.4969
2.1746
0.0010
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

13. Convergence analysis of the Newton’s Method
The Taylor series expansion can be represented as
An approximate solution can be obtained by
At the intersection with x axis f(xi+1)=0, which gives
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

14. Convergence analysis of the Newton’s Method
Substituting the true value xr into Eq (B6.2.1) yields
Subtracting Eq (B6.2.2) from Eq (B6.2.3) yields
Substituting and gives
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

15. Convergence analysis of the Newton’s Method
If we assume convergence, both xi and should eventually approximated by the root xr, and Eq(B6.2.5) can be rearranged to yield
Which means that the current error is proportional to the square of the previous error which indicates quadratic convergence rate.
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

16. Convergence Analysis Remarks
When the guess is close enough to a simple root of the function then Newton’s method is guaranteed to converge quadratically.
Quadratic convergence means that the number of correct digits is nearly doubled at each iteration.
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

17. Problems with Newton’s Method
If the initial guess of the root is far from
the root the method may not converge.
Newton’s method converges linearly near
multiple zeros { f(r) = f ’(r) =0 }. In such a
case modified algorithms can be used to
regain the quadratic convergence.
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

18. Multiple Roots
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

19. Problems with Newton’s Method Runaway
4/10/2017
Problems with Newton’s Method Runaway x0 x1
The estimates of the root is sent away from the root because a near-zero slope is reached
The estimates of the root is going away from the root.
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

20. Problems with Newton’s Method Flat Spot
4/10/2017
Problems with Newton’s Method Flat Spot x0
A zero slope causes division by zero which means that the solution soots off horizontally and never hits the x axis
The value of f’(x) is zero, the algorithm fails.
If f ’(x) is very small then x1 will be very far from x0.
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

21. Problems with Newton’s Method Cycle
4/10/2017
Problems with Newton’s Method Cycle
x1=x3=x5
x0=x2=x4
Cycling between two values happens when an inflection point (f’’(x)=0) occurs in the vicinity of a root.
The algorithm cycles between two values x0 and x1
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

22. Newton’s Method for Systems of nonlinear Equations
4/10/2017
Newton’s Method for Systems of nonlinear Equations
Newton’s Method for Systems of nonlinear Equations
The Newton’s method can be extended to solve system of non-linear equations. To be able to solve systems of non-linear equations, you must be familiar with the following.
Computing derivatives of vector valued functions
Basic matrix calculations (addition, multiplication, determinant,…)
Computing inverse of square matrices
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

23. Example Solve the following system of equations EE3561_Unit 2
(c)AL-DHAIFALLAH1435

24. Solution Using Newton’s Method
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

25. Example Try this Solve the following system of equations EE3561_Unit 2
(c)AL-DHAIFALLAH1435

26. Example Solution
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

27. Secant Method Examples Convergence Analysis
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

28. Newton’s Method (Review)
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

29. Secant Method
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

30. Secant Method
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

31. Secant Method
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

32. Secant Method
Yes NO
Stop
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

33. Modified Secant Method
4/10/2017
Modified Secant Method
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

34. 4/10/2017
Example
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

35. Example x(i) f( x(i) ) x(i+1) |x(i+1)-x(i)| -1.0000 1.0000 -1.1000
0.0585
0.0102
0.0009
0.0001
0.0000
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

36. Convergence Analysis
The rate of convergence of the Secant method is super linear
It is better than Bisection method but not as good as Newton’s method
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

37. Comparison of Root finding methods
Advantages/disadvantages
Examples
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

38. Summary Bisection Newton Secant Reliable, Slow
One function evaluation per iteration
Needs an interval [a,b] containing the root, f(a) f(b)<0
No knowledge of derivative is needed
Newton
Fast (if near the root) but may diverge
Two function evaluation per iteration
Needs derivative and an initial guess x0, f ’ (x0) is nonzero
Secant
Fast (slower than Newton) but may diverge
one function evaluation per iteration
Needs two initial points guess x0, x1 such that
f (x0)- f (x1) is nonzero.
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

39. Example
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

40. Solution _______________________________ k xk f(xk)
________________________________
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

41. Example
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

42. Five iterations of the solution
k xk f(xk) f’(xk) ERROR
______________________________________
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

43. Example
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

44. Example
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

45. Example Estimates of the root of x-cos(x)=0
initial guess
correct digit
correct digits
correct digits
correct digits
EE3561_Unit 2
(c)AL-DHAIFALLAH1435

46. Example In estimating the root of x-cos(x)=0
To get more than 13 correct digits
4 iterations of Newton (x0=0.6)
43 iterations of Bisection method (initial
interval [0.6, .8]
5 iterations of Secant method
( x0=0.6, x1=0.8)
EE3561_Unit 2
(c)AL-DHAIFALLAH1435