1. Ordinary Differential Equations
Equations which are composed of an unknown function and its derivatives are called differential equations.
Differential equations play a fundamental role in engineering because many physical phenomena are best formulated mathematically in terms of their rate of change.
v- dependent variable
t- independent variable

2. Differential equations are also classified as to their order.
When a function involves one dependent variable, the equation is called an ordinary differential equation (or ODE). A partial differential equation (or PDE) involves two or more independent variables.
Differential equations are also classified as to their order.
A first order equation includes a first derivative as its highest derivative.
A second order equation includes a second derivative.
Higher order equations can be reduced to a system of first order equations, by redefining a variable.

3. Classification of ODEs
ODEs can be classified in different ways:
Order
First order ODE
Second order ODE
Nth order ODE
Linearity
Linear ODE
Nonlinear ODE
Auxiliary conditions
Initial value problems
Boundary value problems

4. Analytical Solutions
Analytical Solutions to ODEs are available for linear ODEs and special classes of nonlinear differential equations.

5. Numerical Solutions
Numerical methods are used to obtain a graph or a table of the unknown function.
Most of the Numerical methods used to solve ODEs are based directly (or indirectly) on the truncated Taylor series expansion.

6. Classification of the Numerical Methods
for Solving ODE
Single-Step Methods
Estimates of the solution at a particular step are entirely based on information on the previous step/steps.
Multiple-Step Methods
Estimates of the solution at a particular step are based on information on more than one step

7. Solving Differential Equation
Ordinary D.E.
Partial D.E.
Linear eg.
Nonlinear eg.
Initial value problem
Boundary value problem
Usually no closed-form solution
linearization
numerical solution

8. Discretization in solving D.E.
Errors in Numerical Approach
Discretization error
Stability error y
Exact sol. x
Grid Points

9. Errors
Total error
truncation
round-off
increase as as
trade-off

10. Local error & global error
The error at the given step if it is assumed that all the previous results are all exact
Global error
The true, or accumulated, error

11. Useful concepts(I) Useful concepts in discretization Consistency Order
Convergence

12. Useful concepts(II) stability unstable stable Consistent stable
Converge

13. Stability eg. Exact sol. Euler method For stability
Stability condition
eg.
Exact sol.
Euler method
For stability
Amplification factor

14. Implicit vs. Explicit Method
eg. f
Explicit :
Implicit :
h large y y ye
h small
h increase
explicit t
implicit t
“conditionally stable”
“stable”

15. Auxiliary Conditions Auxiliary Conditions Boundary Conditions
The conditions are not at one point of the independent variable
Initial Conditions
All conditions are at one point of the independent variable

16. Initial value problems
Simultaneous D.E.
High-order D.E.

17. Boundary-Value and Initial value Problems
Boundary-Value Problems
The auxiliary conditions are not at one point of the independent variable
More difficult to solve than initial value problems
Initial-Value Problems
The auxiliary conditions are at one point of the independent variable
same
different

18. Well-posed condition: Dirichlet Theorem

19. Runga-Kutta Methods: Chapter 25
This chapter is devoted to solving ordinary differential equations of the form

20. Taylor series method
Taylor Series Method
Truncation error

21. Example Second order Taylor Series Method

24. Summary of the results:i xi yi
0.00 1
0.01
0.9901 2
0.02
0.9807 3
0.03
0.9716

25. Euler method Taylor series expansion at xo truncation error y ....

26. Figure 25.2

27. The first derivative provides a direct estimate of the slope at xi
where f(xi, yi) is the differential equation evaluated at xi and yi. This estimate can be substituted into the equation:
A new value of y is predicted using the slope to extrapolate linearly over the step size h.

28. Not good

29. Error Analysis for Euler’s Method/
Numerical solutions of ODEs involves two types of error:
Truncation error
Local truncation error
Total or global truncation error “Accumulated truncation error”
Round-off errors

30. Generalizing the relationship
Euler’s approx.
truncation error
Error Analysis
Accumulated truncation error
; 1st order

31. Example; Euler method

32. Maple code of the Previous Example
Resolve for N=100

33. The Taylor series provides a means of quantifying the error in Euler’s method. However;
The Taylor series provides only an estimate of the local truncation error-that is, the error created during a single step of the method.
In actual problems, the functions are more complicated than simple polynomials. Consequently, the derivatives needed to evaluate the Taylor series expansion would not always be easy to obtain.
In conclusion,
the error can be reduced by reducing the step size
If the solution to the differential equation is linear, the method will provide error free predictions as for a straight line the 2nd derivative would be zero.

34. Figure 25.4

35. Improvements of Euler’s method
A fundamental source of error in Euler’s method is that the derivative at the beginning of the interval is assumed to apply across the entire interval.
Three simple modifications are available to circumvent this shortcoming:
Heun’s Method
The Midpoint (or Improved Polygon) Method
Raltson’s Method

36. Heun’s Method/
One method to improve the estimate of the slope involves the determination of two derivatives for the interval:
At the initial point
At the end point
The two derivatives are then averaged to obtain an improved estimate of the slope for the entire interval.

37. Figure 25.9

38. Error analysis; Heun’s Method
Taylor series
Total error
truncation
3rd order
; 2nd order method
Significant improvement over Euler’s method

39. Example; Resolve the same example by Heun’s Methods
improvement

40. Maple code of the Previous Example
Resolve for N=100

41. Runge-Kutta Methods (RK)
Runge-Kutta methods achieve the accuracy of a Taylor series approach without requiring the calculation of higher derivatives.
Increment function
p’s and q’s are constants

42. k’s are recurrence functions
k’s are recurrence functions. Because each k is a functional evaluation, this recurrence makes RK methods efficient for computer calculations.
Various types of RK methods can be devised by employing different number of terms in the increment function as specified by n.
First order RK method with n=1 is in fact Euler’s method.
Once n is chosen, values of a’s, p’s, and q’s are evaluated by setting general equation equal to terms in a Taylor series expansion.

43. Values of a1, a2, p1, and q11 are evaluated by setting the second order equation to Taylor series expansion to the second order term. Three equations to evaluate four unknowns constants are derived.

44. We replace k1 and k2 in to getor
Compare with
and obtain (3 equations-4 unknowns)

45. Huen Method with a Single Corrector (a2=1/2)
Because we can choose an infinite number of values for a2, there are an infinite number of second-order RK methods.
Every version would yield exactly the same results if the solution to ODE were quadratic, linear, or a constant.
However, they yield different results if the solution is more complicated (typically the case).
Three of the most commonly used methods are:
Huen Method with a Single Corrector (a2=1/2)
The Midpoint Method (a2=1)
Raltson’s Method (a2=2/3)

46. Runge-Kutta Methods Three of the most commonly used methods are:
Huen Method with a Single Corrector (a2=1/2)
The Midpoint Method (a2= 1)
Raltson’s Method (a2= 2/3)

47. Runge-Kutta Methods Heun’s Method:y
f(xi,yi) xi
xi+h x
Heun’s Method:
Involves the determination of two derivatives for the interval at the initial point and the end point.
f(xi+h,yi+k1h) y ea xi
xi+h x
Slope: 0.5(k1+k2)

48. Heun’s Method - Example
Obtain a solution between x=0 & 4 (step size = 0.5) for:
Initial conditions are: x = 0 to y = 1,use Heun’s method

49. Runge-Kutta Methods Midpoint Method:y
f(xi,yi) xi
xi+h/2 x
Midpoint Method:
Uses Euler’s method t predict a value of y at the midpoint of the interval:
f(xi+h/2,yi+k1h/2) y ea xi
xi+h x
Slope: k2

50. Midpoint Method (Example 25.6, P705)
Obtain a solution between x=0 & 4 (step size = 0.5) for:
Initial conditions are: x = 0 to y = 1,use Midpoint method

51. Runge-Kutta Methods Ralston’s Method: y f(xi,yi) xi xi+3/4h ea
Slope: (1/3k1+2/3k2)
Ralston’s Method:
f(xi+ 3/4 h, yi+3/4k1h) x
xi+h x

52. Example:

53. Example; Resolve the same example by second order RK methods

54. 3rd Order Runge-Kutta
RK3
CISE301_Topic8L4&5

55. 4-th order Runge-Kutta methods
Fourth-order Runge-Kutta
Taylor series expansion to 4-th order
accurate
short, straight, easy to use P4 P3 P1 P2
※ significant improvement over
modified Euler’s method

56. Higher-Order Runge-Kutta

57. Runge-Kutta method

58. Eg. 4-th order R-K method
Significant
improvement

59. Discussion
Better!

60. Comparison
(5th order)

61. Example

63. One major drawback of the Runge-Kutta methods is that they require more evaluations
of the function f than other methods.

64. Example Euler method for solving a system of first order ODEs.

65. Example : RK2 method for solving a system of first order ODEs

66. Example : RK2 method for solving a system of first order ODEs

67. Methods for Solving a System of First Order ODEs
We have extended Euler and RK2 methods to solve systems of first order ODEs.
Other methods used to solve first order ODE can be easily extended to solve systems of first order ODEs.

68. High Order ODEs How do solve a second order ODE?
How do solve high order ODEs?

69. The General Approach to Solve ODEs
Convert
Solve
High order ODE
System of first order ODEs
Convert
Solve
Second order ODE
Two first order ODEs

70. System of first order ODEs
Conversion Procedure
Convert
Solve
High order ODE
System of first order ODEs
Select the dependent variables
One way is to take the original dependent variable and its derivatives up to one degree less than the highest order derivative.
Write the Differential Equations in terms of the new variables. The equations come from the way the new variables are defined or from the original equation.
Express the equations in a matrix form.

71. Remarks on the Conversion Procedure
Convert
Solve
High order ODE
System of first order ODE
Any nth order ODE is converted to a system of n first order ODEs.
There are an infinite number of ways to select the new variables. As a result, for each high order ODE there are an infinite number of set of equivalent first order systems of ODEs.
Use a table to make the conversion easier.

72. Example of Converting a High Order ODE to First Order ODEs
One degree less than the highest order derivative

73. Example of Converting a High Order ODE to First Order ODEs

74. Example of Converting a High Order ODE to First Order ODEs
One degree less than the highest order derivative

75. Example of Converting a High Order ODE to First Order ODEs

76. Conversion Procedure for Systems of High Order ODEs
Convert
Solve
System of high order ODEs
System of first order ODE
Select the dependent variables
Take the original dependent variables and their derivatives up to one degree less than the highest order derivative for each variable.
Write the Differential Equations in terms of the new variables. The equations come from the way the new variables are defined or from the original equation.
Express the equations in a matrix form.

77. Example of Converting a High Order ODE to First Order ODEs
One degree less than the highest order derivative
One degree less than the highest order derivative

78. Example of Converting a High Order ODE to First Order ODEs

79. Solution of a Second Order ODE
Solve the equation using Euler method. Use h=0.1

80. Solution of a Second Order ODE

81. Summary
Formulas used in solving a first order ODE are used to solve systems of first order ODEs.
Instead of scalar variables and functions, we have vector variables and vector functions.
High order ODEs are converted to a set of first order ODEs.

82. Stiffness and Multistep Methods Chapter 26
Two areas are covered:
Stiff ODEs will be described - ODEs that have both fast and slow components to their solution.
Implicit solution technique and multistep methods will be described.

83. Stiffness
A stiff system is the one involving rapidly changing components together with slowly changing ones.
Both individual and systems of ODEs can be stiff:
If y(0)=0, the analytical solution is developed as:

84. Figure 26.1

85. Insight into the step size required for stability of such a solution can be gained by examining the homogeneous part of the ODE:
The solution starts at y(0)=y0 and asymptotically approaches zero.
If Euler’s method is used to solve the problem numerically:
The stability of this formula depends on the step size h:
solution

86. Thus, for transient part of the equation,
the step size must be < 2/1000 = to maintain stability.
While this criterion maintains stability, an even smaller step size would be required to obtain an accurate solution.
Rather than using explicit approaches, implicit methods offer an alternative remedy. An implicit form of Euler’s method can be developed by evaluating the derivative at a future time.
Backward or implicit Euler’s method:
The approach is called unconditionally stable regardless of the step size 

87. Figure 26.2
Solution by
Explicit
Implicit Euler

88. Multistep Methods The Non-Self-Starting Heun Method
Heun method uses Euler’s method as a predictor and trapezoidal rule as a corrector
Predictor is the weak link in the method because it has the greatest error, O(h2)
One way to improve Heun’s method is to develop a predictor that has a local error of O(h3).

89. Multistep Methods
General Representation of Non-Self-Starting Heun Method

90. Multistep Methods Example 26.2
Integrate y’ = 4e0.8x - 0.5y from x=0 to x=4
(Using h=1 and Initial conditions: x-1 = -1, y-1 = and x0 = 0, y0 = 2)
Solution:
which represents a relative error of % (True value is ). We can apply the Multistep formula iteratively to improve this result:
If we continue the iterations, it converges to y= when ea=-2.68%. Then, for the second step:
With the new method, we get better error rates and faster convergence (compared to the Heun method)

91. Backward or implicit Euler’s method
The approach is called unconditionally stable. Regardless of the step size:

92. Multistep Methods The Non-Self-Starting Heun Method/
Huen method uses Euler’s method as a predictor and trapezoidal rule as a corrector.
Predictor is the weak link in the method because it has the greatest error, O(h2).
One way to improve Heun’s method is to develop a predictor that has a local error of O(h3).

93. Multistep Methods Corrector formula
and you iterate until a maximum number of iterations is reached.

94. Multi-Step Methods 2-Step Methods
In a two-step method, estimates of yi+1 depends on yi, yi-1, xi, and xi-1
xi xi xi xi+1

95. Multi-Step Methods 3-Step Methods
In an 3-step method, estimates of yi+1 depends on yi ,yi-1 ,yi-2, xi , xi-1, and xi-2
xi xi xi xi+1

96. Heun’s Predictor Corrector Method
Heun’s predictor corrector method is not a multi-step method.

97. 2-Step Predictor-Corrector
At each iteration one prediction step is done
and as many correction steps as needed.
is the estimate of the solution at xi+1
after k correction steps.

98. 3-Step Predictor-Corrector

99. 4-Step Adams-Moulton Predictor-Corrector

100. How Many Function Evaluations are Done?
Number of function evaluations is the Computational Speed or Efficiency
How many evaluations per step?
No need to repeat the evaluation of function f at previous points
Only one new function evaluation in the predictor
One function evaluation per correction step
# of function evaluations = 1+ number of corrections

101. Example

102. Example

103. Example

104. Multi-Step Methods Single Step Methods Multistep Methods
Euler and Runge-Kutta are single step methods.
Information about y(x) is used to estimate y(x+h).
Multistep Methods
Adam-Moulton method is a multi-step method.
To estimate y(x+h), information about y(x), y(x-h), y(x-2h)… are used.

105. Number of Steps
At each iteration, one prediction step is done and as many correction steps as needed.
Usually few corrections are done (1 to 3).
It is usually better (in terms of accuracy) to use smaller step size than corrections.

106. Boundary-Value and Eigenvalue Problems Chapter 27
An ODE is accompanied by auxiliary conditions. These conditions are used to evaluate the integral that result during the solution of the equation. An nth order equation requires n conditions.
If all conditions are specified at the same value of the independent variable, then we have an initial-value problem.
If the conditions are specified at different values of the independent variable, usually at extreme points or boundaries of a system, then we have a boundary-value problem.

107. Figure 27.1

108. General Methods for Boundary-value Problems
Figure 27.2

109. (Heat transfer coefficient)
Boundary Conditions
Analytical Solution:

110. The Shooting Method/
Converts the boundary value problem to initial-value problem. A trial-and-error approach is then implemented to solve the initial value approach.
For example, the 2nd order equation can be expressed as two first order ODEs:
An initial value is guessed, say z(0)=10.
The solution is then obtained by integrating the two 1st order ODEs simultaneously.

111. Using a the RK method shown in class.

112. Using Goal seek- one obtains
T(10)=200. for z(0)=
Using a 4th order RK method with a step size of 2:
T(10)=
This differs from T(10)=200. Therefore a new guess is made, z(0)=20 and the computation is performed again.
z(0)=20 T(10)=
Since the two sets of points, (z, T)1 and (z, T)2, are linearly related, a linear interpolation formula is used to compute the value of z(0) as to determine the correct solution.

113. Figure 27.3

114. Learning Objectives
Grasp the difference between initial value problems and boundary value problems.
Appreciate the difficulties involved in solving the boundary value problems.
Grasp the concept of the shooting method.
Use the shooting method to solve boundary value problems.

115. Boundary-Value and Initial Value Problems
Boundary-Value Problems
The auxiliary conditions are not at one point of the independent variable
More difficult to solve than initial value problem
Initial-Value Problems
The auxiliary conditions are at one point of the independent variable
same
different

116. The Shooting Method
Target

117. The Shooting Method
Target

118. The Shooting Method
Target

119. Solution of Boundary-Value Problems Shooting Method for Boundary-Value Problems
Guess a value for the auxiliary conditions at one point of time.
Solve the initial value problem using Euler, Runge-Kutta, …
Check if the boundary conditions are satisfied, otherwise modify the guess and resolve the problem.
Use interpolation in updating the guess.
It is an iterative procedure and can be efficient in solving the BVP.

120. Solution of Boundary-Value Problems Shooting Method
convert
Boundary-Value Problem
Initial-value Problem
Convert the ODE to a system of first order ODEs.
Guess the initial conditions that are not available.
Solve the Initial-value problem.
Check if the known boundary conditions are satisfied.
If needed modify the guess and resolve the problem again.

121. Example Original BVP x

122. Example Original BVP
2. 0 x

123. Example Original BVP
2. 0 x

124. Example 1 Original BVP
2. 0
to be determined x

125. Example 1 Step1: Convert to a System of First Order ODEs

126. Example Guess # 1 x

127. Example Guess # 2
0.99 x

128. Example Interpolation for Guess # 3
y(1)
0.99
Guess
y(1) 1 2
0.9900
y’(0)

129. Example Interpolation for Guess # 3
y(1) 2
Guess 3
0.99
Guess
y(1) 1 2
0.9900
1.5743
y’(0)

130. Example Guess # 3 This is the solution to the boundary value problem.
2.000 x
This is the solution to the boundary value problem.
y(1)=2.000

131. Summary of the Shooting Method
Guess the unavailable values for the auxiliary conditions at one point of the independent variable.
Solve the initial value problem.
Check if the boundary conditions are satisfied, otherwise modify the guess and resolve the problem.
Repeat (3) until the boundary conditions are satisfied.

132. Properties of the Shooting Method
Using interpolation to update the guess often results in few iterations before reaching the solution.
The method can be cumbersome for high order BVP because of the need to guess the initial condition for more than one variable.

133. Finite Difference Method
Use the finite difference method to solve BVP.
Convert linear second order boundary value problems into linear algebraic equations.

134. Solution of Boundary-Value Problems Finite Difference Method
convert
Boundary-Value Problems
Algebraic Equations
Find the unknowns y1, y2, y3
y4=0.8
y3=? y
y1=?
y2=?
y0=0.2 x
x x x x x4

135. Solution of Boundary-Value Problems Finite Difference Method
Divide the interval into n sub-intervals.
The solution of the BVP is converted to the problem of determining the value of function at the base points.
Use finite approximations to replace the derivatives.
This approximation results in a set of algebraic equations.
Solve the equations to obtain the solution of the BVP.

136. Finite Difference Method Example
To be determined
Divide the interval [0,1 ] into n = 4 intervals
Base points are
x0=0
x1=0.25
x2=.5
x3=0.75
x4=1.0
y3=?
y4=0.8 y
y1=?
y2=?
y0=0.2 x
x x x x x4

137. Finite Difference Method Example
Divide the interval [0,1 ] into n = 4 intervals
Base points are
x0=0
x1=0.25
x2=.5
x3=0.75
x4=1.0

138. The Central divided differences
f(x)
f(x)
f(xi+1)
f(xi-1)
f(xi)
f\(xi) Dx Dx x
xi-1 xi
xi+1

139. Second Order BVP

140. Second Order BVP

141. Second Order BVP

142. Second Order BVP

144. Summary of the Discretiztion Methods
Select the base points.
Divide the interval into n sub-intervals.
Use finite approximations to replace the derivatives.
This approximation results in a set of algebraic equations.
Solve the equations to obtain the solution of the BVP.

145. Remarks
Finite Difference Method :
Different formulas can be used for approximating the derivatives.
Different formulas lead to different solutions. All of them are approximate solutions.
For linear second order cases, this reduces to tri-diagonal system.