1. Lecture 5 - Single Variable Problems CVEN 302 June 12, 2002

2. Lecture’s Goals Introduction of Solving Equations of 1 Variable –Bisection Method –Linear interpolation & Secant Method –Newton’s Method –Muller’s Method –Fixed Point Iteration

3. General Considerations Is the function to be evaluated often? How much precision is needed? –If so you may be able to create a custom algorithm for the problem. –Engineering calculations are often required for a few significant figures. A simple root finding procedure may be adequate.

4. General Considerations How fast and robust must the method be? –If the root finding is embedded in another program that automatically changes the parameter f(x), it is important that the root finder is robust. A robust procedure is relatively immune to initial guess and it converges quickly without being overly sensitive to the behavior of the function.

5. General Considerations Is the function a polynomial? –There are special procedures for finding roots of polynomials. These should be used instead of general procedures such as bisection, Newton’s method or secant method.

6. Does the function have singularities? Some root finding procedure will converge to a singularity, as well as converge to a root. This must be guarded against.

7. Basic Root Finding Techniques The initial procedure is to find the roots of a function. The main concept of each technique is to bracket the root and do a series of iteration until the method converges on a solution.

8. Root Finding Techniques Bracket the roots Determine the method Iterate to find the solution

9. Convergence Criteria The algorithm must decide on how close to the root the guess should be before stopping. Two criteria can be applied in testing. –Magnitude by which estimate of root changes. –Magnitude by which the function will change.

10. Bisection Method The method is known as the Bolzano method and can be called interval halving technique. The method is simple and straight-forward. Given a bracketed root, the method repeatedly halves the interval while continuing to bracket the root and it will converge on the solution.

11. Bisection Method (Algorithm) Do while 0.5*|x 1 - x 2 | >= tolerance value Set x 3 =(x 1 + x 2 )/2 IF f(x 3 ) of opposite sign of f(x 1 ); Set x 2 = x 3 ; ELSE Set x 1 = x 3 ; ENDIF END loop

12. Bisection Method DemoBisect: the example program does a simple bisection for a cubic polynomial equation.

13. Bisection Method Example Problem f(x) = a 5 x 5 + a 4 x 4 + a 3 x 3 + a 2 x 2 + a 1 x + a 0 Let the function a 0 = -2, a 1 = -3, a 2 = 4, a 3 = 1, a 4 = 0, a 5 = 1 f(x) = x 5 + x 3 + 4x 2 - 3x - 2

14. Bisection Method There are 3 roots (a) -2 < x < -1 (b) -1 < x < 0 (c) 0.5 < x <1.5

15. Bisection Method x 1 f(x 1 ) x 2 f(x 2 )  x 3 f(x 3 ) -2.0000 -20.000 -1.0000 3.00000 1.0000 -1.5000 0.53125 -2.0000 -20.000 -1.5000 0.53125 0.5000 -1.7500 -6.27246 -1.7500 -6.2725 -1.5000 0.53125 0.2500 -1.6250 -2.18448 -1.6250 -2.1848 -1.5000 0.53125 0.1250 -1.5625 -0.67480 -1.5625 -0.6748 -1.5000 0.53125 0.0625 -1.53125 -0.03612 etc. etc. etc. etc. etc. etc. etc.

16. Linear Interpolation The bisection method is not very efficient. We would like to converge to the root at a faster rate with a different algorithm. One of these method is linear interpolation method or the method of false position (Latinized version Regula Falsi )

17. Method of Linear Interpolation Regula Falsi Do while |x 2 - x 1 | >= tolerance value 1 or |f(x 3 )|>= tolerance value 2 Set x 3 = x 2 - f(x 2 )*(x 2 - x 1 )/(f(x 2 )-f(x 1 )) IF f(x 3 ) of opposite sign of f(x 1 ); Set x 2 = x 3 ; ELSE Set x 1 = x 3 ; ENDIF END loop

18. Regula Flasi or Linear Interpolation The program uses the slope of the two points to find the intersection. However, the upper bound is kept constant. The program uses a similar triangle to estimate the location of the root.

19. Linear Interpolation Same example problem f(x) = x 5 + x 3 + 4x 2 - 3x - 2 roots are between (-1.7,-1.3), (-1,0), & (0.5,1.5)

20. Regula Falsi Method x 1 f(x 1 ) x 2 f(x 2 ) x 2 -x 1 f(x 2 )-f(x 1 ) x 3 f(x 3 ) -1.7000 -4.4516 -1.3000 2.75007 0.4000 7.2016 -1.45275 1.2635 -1.7000 -4.4516 -1.4528 1.2635 0.2472 5.7143 -1.50743 0.40265 -1.7000 -4.4516 -1.5074 0.40265 0.1926 4.8547 -1.52339 0.11307 -1.7000 -4.4516 -1.5234 0.11293 0.1766 4.5645 -1.52777 0.03052 etc. etc. etc. etc. etc. etc. etc. etc.

21. Secant Method The algorithm is similar to the linear interpolation method but it oscillates from one side to the next. The method converges quickly with well behaved functions.

22. Secant Method Do while |x 2 - x 1 | >= tolerance value 1 or |f(x 3 )|>= tolerance value 2 Set x 3 = x 2 - f(x 2 )*(x 2 - x 1 )/(f(x 2 )-f(x 1 )) Set x 1 = x 2 ; Set x 2 = x 3 ; END loop

23. Comparison between Linear Interpolation and Secant Method The secant can be faster method, because it does not have a large slope at the root.

24. Problems with the Secant Method The convergence condition is : x 3 = x 2 - f(x 2 )* (x 2 -x 1 )/(f(x 2 )-f(x 1 )) It is tempting to rewrite the convergence condition. x 3 = (f(x 2 )*x 1 -f(x 1 )* x 2 ) /(f(x 2 )-f(x 1 ))

25. Problems with the Secant Method This method can be catastrophic if the points are near a point where the first derivative is zero. Example: SecantProb Try for the interval (1,3)

26. Secant Method The Secant method is the same as the linear interpolation method, but you do not do a comparison between +/- values and do the substitution. Problem if the points are on opposite ends of a peak or trough.

27. Secant Method x 1 f(x 1 ) x 2 f(x 2 ) x 2  x 1 f(x 2 )-f(x 1 ) x 3 f(x 3 ) -1.7000 -4.4516 -1.3000 2.7501 0.4000 7.2016 -1.45275 1.2635 -1.4528 1.2635 -1.7000 -4.4516 -0.2472 -5.7143 -1.50743 0.40265 -1.5074 0.4031 -1.4528 1.2635 0.0546 0.8596 -1.53300 -0.07000 -1.5330 -0.0700 -1.5074 0.4031 0.0256 0.4731 -1.52921 -0.00297 etc. etc. etc. etc. etc. etc. etc. etc.

28. Newton’s Method This method is one of the most widely used methods to solve equations also known as the Newton-Raphson. The idea for the method comes from a Taylor series expansion, where you know the function and its’ first derivative. f(x k+1 ) = f(x k ) + (x k+1 - x k )*f ‘(x k ) +...

29. Newton’s Method The goal of the calculations is to find a f(x)=0, so set f(x k+1 ) = 0 and rearrange the equation. [ f ‘(x k ) is the first derivative of f(x). ]

30. Newton-Raphson Method The method uses the slope of the line to project to the x axis and find the root. The method converges on the solution quickly.

31. Newton’s Method Do while |x 2 - x 1 | >= tolerance value 1 or |f(x 2 )|>= tolerance value 2 or f’(x 1 ) ~= 0 Set x 2 = x 1 - f(x 1 )/f’(x 1 ) Set x 1 = x 2 ; END loop

32. Newton’s Method Same example problem f(x) = x 5 + x 3 + 4x 2 - 3x - 2 and f’(x) = 5x 4 + 3x 2 + 8x - 3 roots are between (-1.7,-1.3), (-1,0), & (0.5,1.5)

33. Newton’s Method x 1 f(x 1 ) f’(x 1 ) f(x 1 )/f’(x 1 ) x 2 -2.0000 -20.0000 73.0000 -0.27397 -1.7260 -1.7260 -5.3656 36.5037 -0.14699 -1.5790 -1.5790 -1.0423 22.9290 -0.04546 -1.5335 -1.5335 -0.0797 19.4275 -0.00410 -1.5294 -1.5294 -0.0005 19.1381 -0.00003 -1.5294 etc. etc. etc. etc. etc.

34. Summary Bisection - need to know f(x) and two bounds. Regula Flasi (linear interpolation) - need to know the function and two bounds. Secant - need to know f(x) and two bounds. Newton’s - need to know f(x) and f’(x) and an initial guess.

35. Homework Check the Homework webpage