1. M. Dumbser 1 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Lecture on Numerical Analysis Dr.-Ing. Michael Dumbser 08 / 11 / 2007

2. M. Dumbser 2 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Numerical Solution of Nonlinear Equations Task: compute approximately the solution x of the nonlinear algebraic equation If the problem is not in form (1), then it can always be cast into form (1) by putting all terms on the left hand side. Examples: (1)

3. M. Dumbser 3 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Numerical Solution of Nonlinear Equations Task: compute approximately the solution x of the nonlinear algebraic equation (1) Two problems present even at the analytical level: For general functions f(x), there may be no solution at all. Example: For general functions f(x), there may be more than one solution. Example:

4. M. Dumbser 4 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser x0x0 x2x2 x3x3 x1x1 x f x4x4 (I) Before we can start using this method, we must guess two points x 0 and x 1 where the function f(x) has opposite sign, i.e. The Regula Falsi Method (II) Now connect the two points (x 0, f(x 0 )) and (x 1, f(x 1 )) with a straight line and compute the intersection of this line with the x-axis, denoted by x i. The point x i is given by (III) Compute the sign of f(x 2 ) and compare against the signs of f(x 0 ) and f(x 1 ). Then update the interval boundaries according to the result, i.e. set If |f(x i )| < tolerance then STOP (in Maple: break), otherwise go back to step (II).

5. M. Dumbser 5 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser x0x0 x2x2 x3x3 x1x1 x f x4x4 Advantages: - it is guaranteed that the method finds a solution of f(x)=0 because we already know that it the solution is contained in the initial interval [x 0 ;x 1 ]. - The method is very robust and reliable The Regula Falsi Method Disadvantage: - it can not be applied if we do not have an initial guess of the interval [x 0 ;x 1 ].

6. M. Dumbser 6 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser x0x0 x2x2 x1x1 x f x3x3 Variant of the regula falsi method: we start from any two points x 0 and x 1 The Secant Method Advantage: no initial interval must be guessed Disadvantage: may not always converge towards a solution !

7. M. Dumbser 7 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser x0x0 x1x1 x f Variant of the regula falsi method: we start from any two points x 0 and x 1 The Secant Method Advantage: no initial interval must be guessed Disadvantage: may not always converge towards a solution !

8. M. Dumbser 8 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser x0x0 x1x1 x f Differential form of the secant method. Instead of using the secant, Newton proposes to use the tangent! Newtons Method Advantages: - no initial interval must be guessed - very fast Disadvantage: may not always converge towards a solution ! x2x2

9. M. Dumbser 9 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser x0x0 x1x1 x f Newtons Method x2x2 Differential form of the secant method. Instead of using the secant, Newton proposes to use the tangent! (1)If |f(x i )| < tolerance then STOP (in Maple/MATLAB: break) or (2)If | x | < tolerance then STOP

10. M. Dumbser 10 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Exercise 1 Write a Maple worksheet and a MATLAB script that solve the nonlinear equation using the regula falsi method. Choose an appropriate starting interval. Exercise 2 Now write a MATLAB script that solves the problem of exercise 1 using the Newton method. The user should have the possibility to choose the starting guess, the maximum number of iterations as well as the tolerance.

11. M. Dumbser 11 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Globally Convergent Newton Methods From the previous Example 4 we find that even for strictly monotone functions f(x) Newtons method may not always converge from arbitrary starting points x 0, but only from starting points that are sufficiently close to the solution of the problem. However, if the starting point is sufficiently close, the method converges. We therefore say the original Newton method is only locally convergent. The local convergence behaviour of Newtons method and the possible failure to converge for initial guesses far away from the solution can be mainly explained by too large step sizes x:

12. M. Dumbser 12 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Globally Convergent Newton Methods As seen in the sketch of the previous slide, the absolute value of the function f(x) slightly increased during one Newton step. To avoid this problem, we therefore add the following requirement: The function |f(x)| must always decrease during one Newton step, i.e.: If this is not fulfilled, the step size is sucessively reduced until the criterion is satisfied. The globally convergent Newton method then reads as follows: WHILE END DO i=0,1,… UNTIL Free parameter. Choose e.g. = 0.5

13. M. Dumbser 13 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Globally Convergent Newton Methods Exercise 5 Write a modified MATLAB script GlobalNewton.m to solve the nonlinear equation with the globally convergent Newton method. 1)Use x 0 = 2 as initial guess. 2)Use x 0 = 2.5 as initial guess. What do you realize? 3)Use x 0 = 10 6 as initial guess. Does the method converge?

14. M. Dumbser 14 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Computation of the Roots of a Polynomial We now apply Newtons method to a special case of nonlinear algebraic equation where the function f(x) is a polynomial of degree n: The values x which satisfy eqn. (1) are called the roots of the polynomial p n (x). We suppose in the following that all the roots of the polynomial are distinct and real. Suppose x = is a root of p n (x), i.e. then we have If we knew all the roots i we would have the following expression of the polynomial:

15. M. Dumbser 15 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Computation of the Roots of a Polynomial A simple and straightforward application of Newtons method now consists in finding a first root 1 (typically the largest one) of the polynomial p n and then compute the largest root 2 of the polynomial p n-1 obtained by dividing the polynomial p n by ( x – ): After having found the root 2 of p n-1 we divide again and compute the next root 3 and so on, until all the n roots of p n have been found. In general, we have:

16. M. Dumbser 16 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Computation of the Roots of a Polynomial To express the polynomial p n-j (x) and its derivative, two simple MATLAB loops defining the auxiliary variables D j and S j are necessary: with We then obtain the so-called Newton-Maehly method:

17. M. Dumbser 17 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Exercise 3 (1) Write a MATLAB script PolyRoot.m that finds all the roots of a general polynomial of degree n. The user should be able to specify the coefficients a of the polynomial. Then, two MATLAB function files, called Polynomial.m and DPolynomial.m compute the polynomial and its first derivative in point x using the coefficients a. Further user inputs required are tol, NMAX and x0. (2) Apply your general MATLAB script to the particular polynomial (3) Now try your MATLAB program on the polynomial Hint: Use Maple to rewrite this polynomial in standard form

18. M. Dumbser 18 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Efficient Evaluation of Polynomials The standard format of defining a polynomial is computationally not the most efficient way to evaluate a polynomial: A computationally more efficient but mathematically fully equivalent way of writing the polynomial p(x) is the so called Horner scheme, which uses the distributive law: In this example, we have 4 additions and 10 multiplications Now, we still have 4 additions but only 4 multiplications!

19. M. Dumbser 19 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Globally Convergent Newton Methods Exercise 4 Now try your original MATLAB script Newton.m to solve the nonlinear equation 1)Use x 0 = 2 as initial guess. 2)Use x 0 = 2.5 as initial guess. What do you realize?

20. M. Dumbser 20 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Newtons Method for Systems of Equations (1)If ||f(x i )|| < tolerance then STOP (in Maple/MATLAB: break) or (2)If || x || < tolerance then STOP The extension of Newtons method to systems of nonlinear algebraic equations with several unknown variables is straightforward. The problem is written in standard form in matrix-vector notation as follows: Linearization (multi-dimensional Taylor series up to first order) leads to: The matrix D is called the Jacobian matrix of the vector function f with respect to ist vector argument x and is the multi-dimensional equivalent to the first derivative of a scalar function. This corresponds to solving the linear system i.e. the vector norm of f is the so-called function residual.

21. M. Dumbser 21 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Exercise 6 Write a MATLAB script NewtonSys.m that solves the nonlinear system of equations using the Newton method for systems. The user should have the possibility to choose the initial guess for the vector of unknowns, the maximum number of iterations as well as the tolerance. The vector function and its Jacobian matrix should be implemented in two separate.m files, called FuncSys.m and DFuncSys.m.

22. M. Dumbser 22 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Globally Convergent Newtons Method for Systems The extension of the globally convergent Newtons method to systems of nonlinear algebraic equations with several unknown variables is done as in the scalar case: WHILE END DO i=0,1,… UNTIL

23. M. Dumbser 23 / 23 Analisi Numerica Università degli Studi di Trento Dipartimento dIngegneria Civile ed Ambientale Dr.-Ing. Michael Dumbser Exercise 7 We want to solve the following nonlinear system of equations: (1) Use the classical Newton method for systems starting from [0,0] (2) Use the classical Newton method for systems starting from [1000,1000] (3) Can you find a starting point from which the classical Newton method converges? (4) Construct a globally convergent Newton method for systems and start again from the points [0,0] and [1000,1000]. Does the global method even converge from the starting point [1e6,1e6] ?