Presentation is loading. Please wait.

Presentation is loading. Please wait.

Formal Computational Skills Numerical Methods for Differential Equations.

Similar presentations

Presentation on theme: "Formal Computational Skills Numerical Methods for Differential Equations."— Presentation transcript:

1 Formal Computational Skills Numerical Methods for Differential Equations

2 Todays lecture: By the end you should know: How to solve a first order differential equation like: dy/dx = - y + x (eg ctrnn equation) using Eulers method What affects the accuracy of numerical integration The existence of more advanced methods such as Runge-Kutta etc (briefly) how to solve higher order derivatives and issues affecting their solution Why you need to know it: Lots of applications: building agent simulations, simulating neurons, working out robot speeds etc etc

3 Rationale Behind Numerical Methods Differential equations are everywhere in dynamical systems eg: dx/dt = f(t), what is x at t=2 ie x(2)?? Must integrate f(t) Basically: theres lots of difficult maths concerned with exactly integrating differential equations, most of which are impossible Instead: Use some very simple maths to work out an approximate answer which works well enough Trade-off: Can take a LONG time to generate the numerical answer (but also takes a long time to learn hard maths …) God does not care about our mathematical difficulties: he integrates empirically Einstein

4 Difference Equations Basic idea is to go back to the definition of the derivative: If we assume (pretend) dx/dt is constant between t and t+h get: x(t+h) x(t) t t+h Thus if we know: dx/dt = f(t) and assume dx/dt is constant for intervals of h, given x at time t, x(t), we have: x(t+h) = x(t) + h f(t) Or: Eulers method

5 t t+h x(t) x(t+h) t+2h x(t+2h) At t = 1 have x(1) = 2 and dx/dt| t=1 = f(1) = 1 so: dx/dt| t =f(x,t) dx/dt| t+h =f(x,t+h) Eulers method estimates x(t+h) by starting at x(t) … Eg At t=0 have: x(0) = 0 and dx/dt| t=0 = f(0) = 2. Setting h=1: x(1) = x(0) + h dx/dt| t=0 = x(0) + h f(0) = = 2 Basically split time up into bits of h and solve iteratively. Note errors may accumulate then use x(t+h) to get gradient at t+h and use both to estimate x(t+2h) x(2) = x(1) + h dx/dt| t=1 = x(1) + h f(1) = = 3 Then evaluating the gradient at t and x(t).

6 eg: Simulated robot controlled by a neural network which specifies its velocity dx/dt = v(x,t) at each time-step. If it starts at x = 0 at t = 0, what positions will it be in at each time-step? Set step-size h=1 and assume constant v between t and t+h so: Note x must be defined initially in this case at t=0 Therefore known as Eulers method for Initial Value Problems (IVPs) t =01234 … v = … x = x(t+1) = x(t) + v(t) However, positions are only estimates. Robot must accelerate etc so estimates will be wrong. What determines the accuracy?

7 Accuracy Try robot with constant acceleration: evaluate x at t=4 starting with x(0)=0 using: What is the error between estimates at t=4 and real answers? 1) h = 1 t =01234 dx/dt = v = t v = dx/dt =01234 x = 0 [Real ans = 0.5 a t 2 = 0.5 x 16 = 8] 2) double time-step h h = 2 t =024 dx/dt = v = t v = dx/dt =024 x = 0 [Real ans = 8] 3) double acceleration h = 1 t =01234 dx/dt = v = 2t v = dx/dt =02468 x = 0 [Real ans = 16] x(t+h) = x(t) + h dx/dt = x(t) + hv

8 3) double acceleration t =01234 v = dx/dt =02468 x = Error = = 4 So accuracy depends on h AND acceleration ie the smoothness of the function x 1) t =01234 v = dx/dt =01234 x = Error = 8-6 = 2 2) double time-step (h) t =024 v = dx/dt =024 x = 004 Error = 8 – 4 = 4

9 tt+h x(t) x(t+h) h dx/dt Since we assume the gradient is constant over the range h, the accuracy of x(t+h) is increased (in one sense, see later) by making the increase in time-step h (and thus the range of the assumption) small However it also depends on how curvy (non-smooth) x is ie on the size of dx/dt, d 2 x/dt 2, etc as this determines how much changes over h Makes sense intuitively and graphically: Error

10 The reliance of the accuracy of Eulers method on small h can be made explicit by examining the Taylor expansion of y(t) ie: So if : Thus for small h the error is determined by the factor h 2 ie O(h 2 ) and the method is said to be first order accurate But error also depends on higher order differentials so normally assume y is sufficiently smooth that these values are low Error Then the error is:Error =

11 Note that Eulers method is asymmetric as we assume dy/dt has a constant value evaluated at the beginning of the time interval (ie between t and t+h) tt+h y(t) y(t+h) h dy/dt

12 More symmetric (and accurate) to use an estimate from the middle of the interval ie at h/2 tt+h y(t) y(t+h) h dy/dt t+h/2 However to do this we will need to use the value of y at t+h/2 which we do not yet know So use the Runge-Kutta method as follows…

13 1. Estimate y(t+h/2) using dy/dt evaluated at t ie f(y(t),t) (estimate 1) This is the 2 nd Order Runge-Kutta method 2. Use y(t+h/2) to evaluate dy/dt at t+h/2 ie f(y(t+h/2), t+h/2) (estimate 2) 3. Use this estimate of dy/dt to calculate y at t+h starting at y(t) (estimate 3) tt+h y(t+h/2) t+h/2 (estimate 3) Suppose dy/dt= f(y,t) Do: y(t+h/2) = y(t) + h/2 f(y(t), t) (estimate 1) y(t+h) = y(t) + h f(y(t+h/2), t+h/2) (estimate 2)

14 As the name suggests it is 2 nd order accurate (method called nth order if error term is O(h n+1 ) So comparing this estimate with the Taylor expansion: We see the error is O(h 3 ) Why? Can show that: Using a similar procedure but using 4 evaluations of dy/dt we get the commonly used 4 th order Runge-Kutta NB This is the basic idea of how Eulers method is improved: for current/more complicated methods, see Numerical Recipes For MSc, only Runge-Kutta is needed, and usually only Euler

15 Note that high order does NOT always imply high accuracy: rather it means that the method is more efficient ie quicker to run as larger step-sizes can be used Generally, for difficult functions (non-smooth), simpler methods will be more accurate if small h used However, errors also arise as a consequence of rounding in computer evaluation (round-off errors) and while truncation error reduces as we lower h, round-off errors will increase Thus need constraints on step-sizes used so that algorithm remains stable ie round-off errors do not swamp the solution Alternatively, use a combination of forward/backward differences (explicit/implicit) which are much more computationally expensive to run (see Numerical Recipes chapter 19 for more details)

16 or from a previous timestep (backward difference) implicit scheme. More complex to solve Note that one can get an estimate of the gradient at B from an advanced timestep (forward differencing): explicit scheme – fast and simple

17 Higher Order Derivatives For higher order derivatives, can use Euler twice, eg network outputs robot acceleration rather than velocity: But: and so By Euler However, for as the schemes get more complex and higher order, stability becomes more of an issue Also harder to ensure accuracy and so often check it by eg running with time-step h, then time-step h/2 and check difference

Download ppt "Formal Computational Skills Numerical Methods for Differential Equations."

Similar presentations

Ads by Google