Ordinary Differential Equations Equations which are composed of an unknown function and its derivatives are called differential equations. Differential equations play a fundamental role in engineering because many physical phenomena are best formulated mathematically in terms of their rate of change. v- dependent variable t- independent variable

Differential equations are also classified as to their order. When a function involves one dependent variable, the equation is called an ordinary differential equation (or ODE). A partial differential equation (or PDE) involves two or more independent variables. Differential equations are also classified as to their order. A first order equation includes a first derivative as its highest derivative. A second order equation includes a second derivative. Higher order equations can be reduced to a system of first order equations, by redefining a variable.

Classification of ODEs ODEs can be classified in different ways: Order First order ODE Second order ODE Nth order ODE Linearity Linear ODE Nonlinear ODE Auxiliary conditions Initial value problems Boundary value problems

Analytical Solutions Analytical Solutions to ODEs are available for linear ODEs and special classes of nonlinear differential equations.

Numerical Solutions Numerical methods are used to obtain a graph or a table of the unknown function. Most of the Numerical methods used to solve ODEs are based directly (or indirectly) on the truncated Taylor series expansion.

Classification of the Numerical Methods for Solving ODE Single-Step Methods Estimates of the solution at a particular step are entirely based on information on the previous step/steps. Multiple-Step Methods Estimates of the solution at a particular step are based on information on more than one step

Solving Differential Equation Ordinary D.E. Partial D.E. Linear eg. Nonlinear eg. Initial value problem Boundary value problem Usually no closed-form solution linearization numerical solution

Discretization in solving D.E. Errors in Numerical Approach Discretization error Stability error y Exact sol. x Grid Points

Errors Total error truncation round-off increase as as trade-off

Local error & global error The error at the given step if it is assumed that all the previous results are all exact Global error The true, or accumulated, error

Useful concepts(I) Useful concepts in discretization Consistency Order Convergence

Useful concepts(II) stability unstable stable Consistent stable Converge

Stability eg. Exact sol. Euler method For stability Stability condition eg. Exact sol. Euler method For stability Amplification factor

Implicit vs. Explicit Method eg. f Explicit : Implicit : h large y y ye h small h increase explicit t implicit t “conditionally stable” “stable”

Auxiliary Conditions Auxiliary Conditions Boundary Conditions The conditions are not at one point of the independent variable Initial Conditions All conditions are at one point of the independent variable

Initial value problems Simultaneous D.E. High-order D.E.

Boundary-Value and Initial value Problems Boundary-Value Problems The auxiliary conditions are not at one point of the independent variable More difficult to solve than initial value problems Initial-Value Problems The auxiliary conditions are at one point of the independent variable same different

Well-posed condition: Dirichlet Theorem

Runga-Kutta Methods: Chapter 25 This chapter is devoted to solving ordinary differential equations of the form

Taylor series method Taylor Series Method Truncation error

Example Second order Taylor Series Method

Summary of the results: i xi yi 0.00 1 0.01 0.9901 2 0.02 0.9807 3 0.03 0.9716

Euler method Taylor series expansion at xo truncation error y ....

Figure 25.2

The first derivative provides a direct estimate of the slope at xi where f(xi, yi) is the differential equation evaluated at xi and yi. This estimate can be substituted into the equation: A new value of y is predicted using the slope to extrapolate linearly over the step size h.

Not good

Error Analysis for Euler’s Method/ Numerical solutions of ODEs involves two types of error: Truncation error Local truncation error Total or global truncation error “Accumulated truncation error” Round-off errors

Generalizing the relationship Euler’s approx. truncation error Error Analysis Accumulated truncation error ; 1st order

Example; Euler method

Maple code of the Previous Example Resolve for N=100

The Taylor series provides a means of quantifying the error in Euler’s method. However; The Taylor series provides only an estimate of the local truncation error-that is, the error created during a single step of the method. In actual problems, the functions are more complicated than simple polynomials. Consequently, the derivatives needed to evaluate the Taylor series expansion would not always be easy to obtain. In conclusion, the error can be reduced by reducing the step size If the solution to the differential equation is linear, the method will provide error free predictions as for a straight line the 2nd derivative would be zero.

Figure 25.4

Improvements of Euler’s method A fundamental source of error in Euler’s method is that the derivative at the beginning of the interval is assumed to apply across the entire interval. Three simple modifications are available to circumvent this shortcoming: Heun’s Method The Midpoint (or Improved Polygon) Method Raltson’s Method

Heun’s Method/ One method to improve the estimate of the slope involves the determination of two derivatives for the interval: At the initial point At the end point The two derivatives are then averaged to obtain an improved estimate of the slope for the entire interval.

Figure 25.9

Error analysis; Heun’s Method Taylor series Total error truncation 3rd order ; 2nd order method Significant improvement over Euler’s method

Example; Resolve the same example by Heun’s Methods improvement

Maple code of the Previous Example Resolve for N=100

Runge-Kutta Methods (RK) Runge-Kutta methods achieve the accuracy of a Taylor series approach without requiring the calculation of higher derivatives. Increment function p’s and q’s are constants

k’s are recurrence functions k’s are recurrence functions. Because each k is a functional evaluation, this recurrence makes RK methods efficient for computer calculations. Various types of RK methods can be devised by employing different number of terms in the increment function as specified by n. First order RK method with n=1 is in fact Euler’s method. Once n is chosen, values of a’s, p’s, and q’s are evaluated by setting general equation equal to terms in a Taylor series expansion.

Values of a1, a2, p1, and q11 are evaluated by setting the second order equation to Taylor series expansion to the second order term. Three equations to evaluate four unknowns constants are derived.

We replace k1 and k2 in to get or Compare with and obtain (3 equations-4 unknowns)

Huen Method with a Single Corrector (a2=1/2) Because we can choose an infinite number of values for a2, there are an infinite number of second-order RK methods. Every version would yield exactly the same results if the solution to ODE were quadratic, linear, or a constant. However, they yield different results if the solution is more complicated (typically the case). Three of the most commonly used methods are: Huen Method with a Single Corrector (a2=1/2) The Midpoint Method (a2=1) Raltson’s Method (a2=2/3)

Runge-Kutta Methods Three of the most commonly used methods are: Huen Method with a Single Corrector (a2=1/2) The Midpoint Method (a2= 1) Raltson’s Method (a2= 2/3)

Runge-Kutta Methods Heun’s Method: y f(xi,yi) xi xi+h x Heun’s Method: Involves the determination of two derivatives for the interval at the initial point and the end point. f(xi+h,yi+k1h) y ea xi xi+h x Slope: 0.5(k1+k2)

Heun’s Method - Example Obtain a solution between x=0 & 4 (step size = 0.5) for: Initial conditions are: x = 0 to y = 1,use Heun’s method

Runge-Kutta Methods Midpoint Method: y f(xi,yi) xi xi+h/2 x Midpoint Method: Uses Euler’s method t predict a value of y at the midpoint of the interval: f(xi+h/2,yi+k1h/2) y ea xi xi+h x Slope: k2

Midpoint Method (Example 25.6, P705) Obtain a solution between x=0 & 4 (step size = 0.5) for: Initial conditions are: x = 0 to y = 1,use Midpoint method

Runge-Kutta Methods Ralston’s Method: y f(xi,yi) xi xi+3/4h ea Slope: (1/3k1+2/3k2) Ralston’s Method: f(xi+ 3/4 h, yi+3/4k1h) x xi+h x

Example:

Example; Resolve the same example by second order RK methods

3rd Order Runge-Kutta RK3 CISE301_Topic8L4&5

4-th order Runge-Kutta methods Fourth-order Runge-Kutta Taylor series expansion to 4-th order accurate short, straight, easy to use P4 P3 P1 P2 ※ significant improvement over modified Euler’s method

Higher-Order Runge-Kutta

Runge-Kutta method

Eg. 4-th order R-K method Significant improvement

Discussion Better!

Comparison (5th order)

Example

One major drawback of the Runge-Kutta methods is that they require more evaluations of the function f than other methods.

Example Euler method for solving a system of first order ODEs.

Example : RK2 method for solving a system of first order ODEs

Example : RK2 method for solving a system of first order ODEs

Methods for Solving a System of First Order ODEs We have extended Euler and RK2 methods to solve systems of first order ODEs. Other methods used to solve first order ODE can be easily extended to solve systems of first order ODEs.

High Order ODEs How do solve a second order ODE? How do solve high order ODEs?

The General Approach to Solve ODEs Convert Solve High order ODE System of first order ODEs Convert Solve Second order ODE Two first order ODEs

System of first order ODEs Conversion Procedure Convert Solve High order ODE System of first order ODEs Select the dependent variables One way is to take the original dependent variable and its derivatives up to one degree less than the highest order derivative. Write the Differential Equations in terms of the new variables. The equations come from the way the new variables are defined or from the original equation. Express the equations in a matrix form.

Remarks on the Conversion Procedure Convert Solve High order ODE System of first order ODE Any nth order ODE is converted to a system of n first order ODEs. There are an infinite number of ways to select the new variables. As a result, for each high order ODE there are an infinite number of set of equivalent first order systems of ODEs. Use a table to make the conversion easier.

Example of Converting a High Order ODE to First Order ODEs One degree less than the highest order derivative

Example of Converting a High Order ODE to First Order ODEs

Example of Converting a High Order ODE to First Order ODEs One degree less than the highest order derivative

Example of Converting a High Order ODE to First Order ODEs

Conversion Procedure for Systems of High Order ODEs Convert Solve System of high order ODEs System of first order ODE Select the dependent variables Take the original dependent variables and their derivatives up to one degree less than the highest order derivative for each variable. Write the Differential Equations in terms of the new variables. The equations come from the way the new variables are defined or from the original equation. Express the equations in a matrix form.

Example of Converting a High Order ODE to First Order ODEs One degree less than the highest order derivative One degree less than the highest order derivative

Example of Converting a High Order ODE to First Order ODEs

Solution of a Second Order ODE Solve the equation using Euler method. Use h=0.1

Solution of a Second Order ODE

Summary Formulas used in solving a first order ODE are used to solve systems of first order ODEs. Instead of scalar variables and functions, we have vector variables and vector functions. High order ODEs are converted to a set of first order ODEs.

Stiffness and Multistep Methods Chapter 26 Two areas are covered: Stiff ODEs will be described - ODEs that have both fast and slow components to their solution. Implicit solution technique and multistep methods will be described.

Stiffness A stiff system is the one involving rapidly changing components together with slowly changing ones. Both individual and systems of ODEs can be stiff: If y(0)=0, the analytical solution is developed as:

Figure 26.1

Insight into the step size required for stability of such a solution can be gained by examining the homogeneous part of the ODE: The solution starts at y(0)=y0 and asymptotically approaches zero. If Euler’s method is used to solve the problem numerically: The stability of this formula depends on the step size h: solution

Thus, for transient part of the equation, the step size must be < 2/1000 = 0.002 to maintain stability. While this criterion maintains stability, an even smaller step size would be required to obtain an accurate solution. Rather than using explicit approaches, implicit methods offer an alternative remedy. An implicit form of Euler’s method can be developed by evaluating the derivative at a future time. Backward or implicit Euler’s method: The approach is called unconditionally stable regardless of the step size 

Figure 26.2 Solution by Explicit Implicit Euler

Multistep Methods The Non-Self-Starting Heun Method Heun method uses Euler’s method as a predictor and trapezoidal rule as a corrector Predictor is the weak link in the method because it has the greatest error, O(h2) One way to improve Heun’s method is to develop a predictor that has a local error of O(h3).

Multistep Methods General Representation of Non-Self-Starting Heun Method

Multistep Methods Example 26.2 Integrate y’ = 4e0.8x - 0.5y from x=0 to x=4 (Using h=1 and Initial conditions: x-1 = -1, y-1 = -0.393 and x0 = 0, y0 = 2) Solution: which represents a relative error of -5.73% (True value is 6.1946). We can apply the Multistep formula iteratively to improve this result: If we continue the iterations, it converges to y=6.36086 when ea=-2.68%. Then, for the second step: With the new method, we get better error rates and faster convergence (compared to the Heun method)

Backward or implicit Euler’s method The approach is called unconditionally stable. Regardless of the step size:

Multistep Methods The Non-Self-Starting Heun Method/ Huen method uses Euler’s method as a predictor and trapezoidal rule as a corrector. Predictor is the weak link in the method because it has the greatest error, O(h2). One way to improve Heun’s method is to develop a predictor that has a local error of O(h3).

Multistep Methods Corrector formula and you iterate until a maximum number of iterations is reached.

Multi-Step Methods 2-Step Methods In a two-step method, estimates of yi+1 depends on yi, yi-1, xi, and xi-1 xi-2 xi-1 xi xi+1

Multi-Step Methods 3-Step Methods In an 3-step method, estimates of yi+1 depends on yi ,yi-1 ,yi-2, xi , xi-1, and xi-2 xi-2 xi-1 xi xi+1

Heun’s Predictor Corrector Method Heun’s predictor corrector method is not a multi-step method.

2-Step Predictor-Corrector At each iteration one prediction step is done and as many correction steps as needed. is the estimate of the solution at xi+1 after k correction steps.

3-Step Predictor-Corrector

4-Step Adams-Moulton Predictor-Corrector

How Many Function Evaluations are Done? Number of function evaluations is the Computational Speed or Efficiency How many evaluations per step? No need to repeat the evaluation of function f at previous points Only one new function evaluation in the predictor One function evaluation per correction step # of function evaluations = 1+ number of corrections

Example

Example

Example

Multi-Step Methods Single Step Methods Multistep Methods Euler and Runge-Kutta are single step methods. Information about y(x) is used to estimate y(x+h). Multistep Methods Adam-Moulton method is a multi-step method. To estimate y(x+h), information about y(x), y(x-h), y(x-2h)… are used.

Number of Steps At each iteration, one prediction step is done and as many correction steps as needed. Usually few corrections are done (1 to 3). It is usually better (in terms of accuracy) to use smaller step size than corrections.

Boundary-Value and Eigenvalue Problems Chapter 27 An ODE is accompanied by auxiliary conditions. These conditions are used to evaluate the integral that result during the solution of the equation. An nth order equation requires n conditions. If all conditions are specified at the same value of the independent variable, then we have an initial-value problem. If the conditions are specified at different values of the independent variable, usually at extreme points or boundaries of a system, then we have a boundary-value problem.

Figure 27.1

General Methods for Boundary-value Problems Figure 27.2

(Heat transfer coefficient) Boundary Conditions Analytical Solution:

The Shooting Method/ Converts the boundary value problem to initial-value problem. A trial-and-error approach is then implemented to solve the initial value approach. For example, the 2nd order equation can be expressed as two first order ODEs: An initial value is guessed, say z(0)=10. The solution is then obtained by integrating the two 1st order ODEs simultaneously.

Using a the RK method shown in class.

Using Goal seek- one obtains T(10)=200. for z(0)=18.11654 Using a 4th order RK method with a step size of 2: T(10)=168.3797. This differs from T(10)=200. Therefore a new guess is made, z(0)=20 and the computation is performed again. z(0)=20 T(10)=285.8980 Since the two sets of points, (z, T)1 and (z, T)2, are linearly related, a linear interpolation formula is used to compute the value of z(0) as 12.6907 to determine the correct solution.

Figure 27.3

Learning Objectives Grasp the difference between initial value problems and boundary value problems. Appreciate the difficulties involved in solving the boundary value problems. Grasp the concept of the shooting method. Use the shooting method to solve boundary value problems.

Boundary-Value and Initial Value Problems Boundary-Value Problems The auxiliary conditions are not at one point of the independent variable More difficult to solve than initial value problem Initial-Value Problems The auxiliary conditions are at one point of the independent variable same different

The Shooting Method Target

The Shooting Method Target

The Shooting Method Target

Solution of Boundary-Value Problems Shooting Method for Boundary-Value Problems Guess a value for the auxiliary conditions at one point of time. Solve the initial value problem using Euler, Runge-Kutta, … Check if the boundary conditions are satisfied, otherwise modify the guess and resolve the problem. Use interpolation in updating the guess. It is an iterative procedure and can be efficient in solving the BVP.

Solution of Boundary-Value Problems Shooting Method convert Boundary-Value Problem Initial-value Problem Convert the ODE to a system of first order ODEs. Guess the initial conditions that are not available. Solve the Initial-value problem. Check if the known boundary conditions are satisfied. If needed modify the guess and resolve the problem again.

Example Original BVP 0 1 x

Example Original BVP 2. 0 0 1 x

Example Original BVP 2. 0 0 1 x

Example 1 Original BVP 2. 0 to be determined 0 1 x

Example 1 Step1: Convert to a System of First Order ODEs

Example Guess # 1 -0.7688 0 1 x

Example Guess # 2 0.99 0 1 x

Example Interpolation for Guess # 3 y(1) 0.99 Guess y(1) 1 -0.7688 2 0.9900 0 1 2 y’(0) -0.7688

Example Interpolation for Guess # 3 y(1) 2 Guess 3 0.99 Guess y(1) 1 -0.7688 2 0.9900 1.5743 0 1 2 y’(0) -0.7688

Example Guess # 3 This is the solution to the boundary value problem. 2.000 0 1 x This is the solution to the boundary value problem. y(1)=2.000

Summary of the Shooting Method Guess the unavailable values for the auxiliary conditions at one point of the independent variable. Solve the initial value problem. Check if the boundary conditions are satisfied, otherwise modify the guess and resolve the problem. Repeat (3) until the boundary conditions are satisfied.

Properties of the Shooting Method Using interpolation to update the guess often results in few iterations before reaching the solution. The method can be cumbersome for high order BVP because of the need to guess the initial condition for more than one variable.

Finite Difference Method Use the finite difference method to solve BVP. Convert linear second order boundary value problems into linear algebraic equations.

Solution of Boundary-Value Problems Finite Difference Method convert Boundary-Value Problems Algebraic Equations Find the unknowns y1, y2, y3 y4=0.8 y3=? y y1=? y2=? y0=0.2 0 0.25 0.5 0.75 1.0 x x0 x1 x2 x3 x4

Solution of Boundary-Value Problems Finite Difference Method Divide the interval into n sub-intervals. The solution of the BVP is converted to the problem of determining the value of function at the base points. Use finite approximations to replace the derivatives. This approximation results in a set of algebraic equations. Solve the equations to obtain the solution of the BVP.

Finite Difference Method Example To be determined Divide the interval [0,1 ] into n = 4 intervals Base points are x0=0 x1=0.25 x2=.5 x3=0.75 x4=1.0 y3=? y4=0.8 y y1=? y2=? y0=0.2 0 0.25 0.5 0.75 1.0 x x0 x1 x2 x3 x4

Finite Difference Method Example Divide the interval [0,1 ] into n = 4 intervals Base points are x0=0 x1=0.25 x2=.5 x3=0.75 x4=1.0

The Central divided differences f(x) f(x) f(xi+1) f(xi-1) f(xi) f\(xi) Dx Dx x xi-1 xi xi+1

Second Order BVP

Second Order BVP

Second Order BVP

Second Order BVP

Summary of the Discretiztion Methods Select the base points. Divide the interval into n sub-intervals. Use finite approximations to replace the derivatives. This approximation results in a set of algebraic equations. Solve the equations to obtain the solution of the BVP.

Remarks Finite Difference Method : Different formulas can be used for approximating the derivatives. Different formulas lead to different solutions. All of them are approximate solutions. For linear second order cases, this reduces to tri-diagonal system.