Running a model's adjoint to obtain derivatives, while more efficient and accurate than other methods, such as the finite difference method, is a computationally.

Slides:



Advertisements
Similar presentations
Quality control tools
Advertisements

Bayesian tools for analysing and reducing uncertainty Tony OHagan University of Sheffield.
Overview of Lecture Partitioning Evaluating the Null Hypothesis ANOVA
SADC Course in Statistics Introduction to Statistical Inference (Session 03)
Example Project and Numerical Integration Computational Neuroscience 03 Lecture 11.
CHAPTER 11: Sampling Distributions
An Introduction to Matching and Layout Alan Hastings Texas Instruments
Lecture 32 APPLICATIONS OF BIVARIATE OPTIMIZATION.
Using an emulator. Outline So we’ve built an emulator – what can we use it for? Prediction What would the simulator output y be at an untried input x.
Insert Date HereSlide 1 Using Derivative and Integral Information in the Statistical Analysis of Computer Models Gemma Stephenson March 2007.
Copyright © Cengage Learning. All rights reserved.
SOLUTION OF STATE EQUATION
TECHNIQUES OF INTEGRATION
Regression Analysis Once a linear relationship is defined, the independent variable can be used to forecast the dependent variable. Y ^ = bo + bX bo is.
« هو اللطیف » By : Atefe Malek. khatabi Spring 90.
Regression Analysis Using Excel. Econometrics Econometrics is simply the statistical analysis of economic phenomena Here, we just summarize some of the.
Algebra Problems… Solutions Algebra Problems… Solutions © 2007 Herbert I. Gross Set 4 By Herb I. Gross and Richard A. Medeiros next.
Climate case study. Outline The challenge The simulator The data Definitions and conventions Elicitation Expert beliefs about climate parameters Expert.
8 TECHNIQUES OF INTEGRATION. In defining a definite integral, we dealt with a function f defined on a finite interval [a, b] and we assumed that f does.
INTEGRALS 5. INTEGRALS We saw in Section 5.1 that a limit of the form arises when we compute an area.  We also saw that it arises when we try to find.
Evaluating Hypotheses
APPLICATIONS OF INTEGRATION
The Calibration Process
Lehrstuhl für Informatik 2 Gabriella Kókai: Maschine Learning 1 Evaluating Hypotheses.
8 TECHNIQUES OF INTEGRATION. There are two situations in which it is impossible to find the exact value of a definite integral. TECHNIQUES OF INTEGRATION.
In the previous two sections, we focused on finding solutions to differential equations. However, most differential equations cannot be solved explicitly.
Ch 8.1 Numerical Methods: The Euler or Tangent Line Method
Boyce/DiPrima 9th ed, Ch 8.4: Multistep Methods Elementary Differential Equations and Boundary Value Problems, 9th edition, by William E. Boyce and Richard.
Integrals 5.
Ch 8.3: The Runge-Kutta Method
APPLICATIONS OF INTEGRATION 6. A= Area between f and g Summary In general If.
SECOND-ORDER DIFFERENTIAL EQUATIONS Series Solutions SECOND-ORDER DIFFERENTIAL EQUATIONS In this section, we will learn how to solve: Certain.
Statistics for Managers Using Microsoft Excel, 5e © 2008 Pearson Prentice-Hall, Inc.Chap 8-1 Statistics for Managers Using Microsoft® Excel 5th Edition.
LECTURER PROF.Dr. DEMIR BAYKA AUTOMOTIVE ENGINEERING LABORATORY I.
Using an emulator. Outline So we’ve built an emulator – what can we use it for? Prediction What would the simulator output y be at an untried input x?
3 DERIVATIVES. The functions that we have met so far can be described by expressing one variable explicitly in terms of another variable.  For example,,
Integrals  In Chapter 2, we used the tangent and velocity problems to introduce the derivative—the central idea in differential calculus.  In much the.
Copyright © Cengage Learning. All rights reserved. 3 Introduction to the Derivative.
17 May 2007RSS Kent Local Group1 Quantifying uncertainty in the UK carbon flux Tony O’Hagan CTCD, Sheffield.
Center for Radiative Shock Hydrodynamics Fall 2011 Review Assessment of predictive capability Derek Bingham 1.
Limits and Derivatives
CFD Refinement By: Brian Cowley. Overview 1.Background on CFD 2.How it works 3.CFD research group on campus for which problem exists o Our current techniques.
Problem Transient Response By: Matt Lausmann ME 114 Granda Assignment #3.
In section 11.9, we were able to find power series representations for a certain restricted class of functions. Here, we investigate more general problems.
TECHNIQUES OF INTEGRATION Due to the Fundamental Theorem of Calculus (FTC), we can integrate a function if we know an antiderivative, that is, an indefinite.
- 1 - Overall procedure of validation Calibration Validation Figure 12.4 Validation, calibration, and prediction (Oberkampf and Barone, 2004 ). Model accuracy.
8 TECHNIQUES OF INTEGRATION. Due to the Fundamental Theorem of Calculus (FTC), we can integrate a function if we know an antiderivative, that is, an indefinite.
Advanced Engineering Mathematics, 7 th Edition Peter V. O’Neil © 2012 Cengage Learning Engineering. All Rights Reserved. CHAPTER 4 Series Solutions.
In Chapters 6 and 8, we will see how to use the integral to solve problems concerning:  Volumes  Lengths of curves  Population predictions  Cardiac.
Ch 8.2: Improvements on the Euler Method Consider the initial value problem y' = f (t, y), y(t 0 ) = y 0, with solution  (t). For many problems, Euler’s.
INTEGRALS We saw in Section 5.1 that a limit of the form arises when we compute an area. We also saw that it arises when we try to find the distance traveled.
5 INTEGRALS.
Topics 1 Specific topics to be covered are: Discrete-time signals Z-transforms Sampling and reconstruction Aliasing and anti-aliasing filters Sampled-data.
Lecture 39 Numerical Analysis. Chapter 7 Ordinary Differential Equations.
6.1 Areas Between Curves In this section we learn about: Using integrals to find areas of regions that lie between the graphs of two functions. APPLICATIONS.
Introduction to emulators Tony O’Hagan University of Sheffield.
5-2-3 Analogue to Digital Converters (ADC). Analogue to Digital Conversion The process is now the opposite of that studied in Topic Now we wish.
8 Sept 2006, DEMA2006Slide 1 An Introduction to Computer Experiments and their Design Problems Tony O’Hagan University of Sheffield.
In this chapter, we explore some of the applications of the definite integral by using it to compute areas between curves, volumes of solids, and the work.
INTRODUCTION : Convection: Heat transfer between a solid surface and a moving fluid is governed by the Newton’s cooling law: q = hA(Ts-Tɷ), where Ts is.
Constructive Cost Model
Modeling the Atmos.-Ocean System
Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.
6.2 Grid Search of Chi-Square Space
Further Topics on Random Variables: Derived Distributions
Further Topics on Random Variables: Derived Distributions
Propagation of Error Berlin Chen
Propagation of Error Berlin Chen
Further Topics on Random Variables: Derived Distributions
Presentation transcript:

Running a model's adjoint to obtain derivatives, while more efficient and accurate than other methods, such as the finite difference method, is a computationally expensive task. In addition, the effort taken initially to write the adjoint is considerable and the task, time consuming. Therefore, if we can emulate the derivatives of a model this would decrease the demand for writing and running adjoints. The derivatives of a Gaussian process remain a Gaussian process and we are able therefore to model the derivatives of a model as a Gaussian process. Some performances of emulators to predict derivatives, built with and without derivatives, are shown below in Figures Figure 10 (left). Emulating derivatives based on function output and known derivatives at 5 points. Figure 10 shows the performance of an emulator predicting the derivatives of the the true model, x + cos(x) + sin(x), based on the model output and the derivatives at 5 points. Figure 11 (left). Emulating derivatives based on function output at 5 points. Figure 11 shows the performance of an emulator based on training data which consists only of the simulator output at the 5 design points. We can see from Figure 11 that though we can still predict the derivatives of the true function, the uncertainty is much greater and the posterior mean is further from the true value than in the emulator which had derivative information included in the training data. It should be noted, though, that the computational expense required to build the two emulators of Figures 10 and 11 are not equal. Figure 12 (right). Emulating derivatives based on function output at 8 points. It would appear from Figure 11 that without derivative information in the training data, 5 runs of this simulator aren't enough for the emulator mean to produce an adequate approximation. Continuing with the example, we evaluate 3 more runs of the simulator and this yields an emulator, of which the mean and standard deviation is shown in Figure 12. We may not have a model’s adjoint and as such, the derivatives are unknown. We can still, however, emulate the derivatives of the model. Figures 11 and 12 illustrate emulators built only with function output. Using Derivative Information in the Statistical Analysis of Computer Models G. Stephenson, P. Challenor, R. Marsh 1. Introduction to Gaussian Process Emulators 6. Conclusions Thanks to Jeremy Oakley, Robin Hankin and all in the MUCM team. www. mucm.group.shef.ac.uk 4. Emulation of Derivatives Complex computer models are used in many areas, such as engineering and environmental science, to simulate the behaviour of real-world systems. An appreciable amount of computing time may be required to run a complex model and performing analyses such as sensitivity and uncertainty analysis can require many runs of the simulator. This quickly becomes impractical with a computationally expensive model. Greater efficiency can be achieved by building an emulator, which is a statistical approximation to the simulator. The approach we use is to model the simulator with a Gaussian process. The emulator is built based on data collected from running the simulator at a specified, small number of input points. A Gaussian process emulator is illustrated below, in Figure 1. Figure 1 (left). The true function is evaluated at 5 points and training on this output, an emulator built. The emulator mean is then evaluated at a set of untried input points and the standard deviation at each of these points calculated. In this example, the posterior mean is close to the true value of the simulator; the predictions become worse and uncertainty much greater once it is forced to extrapolate. At the 5 design points the uncertainty pinches in to zero, as the true value of the simulator at these points is known. The uncertainty becomes more appreciable the further away from a design point we predict at, how quickly the uncertainty grows between design points depends on the roughness parameter in the emulator. If we increase the number of design points we would expect the predictions to become closer to the simulator output and the uncertainty to decrease. Figure 2 (left). Emulator with derivative information at 5 points. The emulator in Figure 1 is repeated but here, in addition to the function output, we include the derivatives of the output w.r.t. the input at each of the 5 points. The emulator mean is again evaluated at a set of untried input points and the standard deviation at each of these points calculated. The resulting emulator is shown in Figure 2. The Figure 3 (above, right). Emulator with derivative information at 3 points. The posterior mean is still close to the true simulator output. The uncertainty reduces to zero at design points, as expected, but whereas in Figure 1 the uncertainty becomes appreciable once we start predicting away from a design point, here the uncertainty remains very small for predictions closes to the design points. It is the derivative information in the model which allows for this reduced uncertainty. It is necessary to consider the computational cost of using derivative information in computer experiments as it must be determined at which point the costs outweigh the benefits. For example, if generating the derivatives of the model increases the computational cost substantially, this extra computing time may be better spent evaluating the model at more points instead. 3. Toy Model Investigation C-GOLDSTEIN is an intermediate complexity climate model and the adjoint of the model exists enabling the evaluation of derivatives. An example of the type of output C-GOLDSTEIN produces is shown in Figure 13. Gaussian process emulation can provide a practical solution to running computationally expensive models and results from investigations on toy models show that using derivative information when building emulators could improve efficiency. This is, however, dependent on the computational cost of obtaining derivatives. Moreover, we can use an emulator to predict the derivatives of a complex model. Further investigation is required, but current work suggests that emulation of derivatives could reduce the demand for writing and running adjoint models. Work will continue by further emulation of both the function output and derivatives of C-GOLDSTEIN. It is possible to obtain derivatives of model outputs with respect to its inputs. One approach is using Automatic Differentiation (AD). Derivatives are generated by repeatedly applying the chain rule to the combinations of elementary operations in the model. The differentiated code then runs along side the original code in the resulting model, which is termed the adjoint model. The value of learning derivatives when building emulators is being investigated to determine whether additional e ffi ciency can be achieved. Figures 2 and 3 illustrate a Gaussian process emulator which has been built with the additional information provided by derivatives. The performance of the emulator in Figure 2, while good, is such that it is difficult to identify precisely how and where the derivative information is having an effect. Due to this we now repeat the example but remove two of the simulator runs and corresponding derivatives from the training data. We emulate each toy model, with and without derivative information, for a range of simulator runs. As the simulators here are not complex models, it is possible to run them at all points we test the emulator at. An average prediction error is then determined by looking at the difference between the value of the posterior mean, and the true value at that point as given by the simulator. We investigate the value of derivative information in Gaussian process emulation with 3 toy models. These are shown below in Figures Figure 7 - Performance of Emulator 1Figure 8 - Performance of Emulator 2Figure 9 - Performance of Emulator 3 Figures show that for all toy models tested here, the mean of emulators built with derivatives provide a closer approximation to the relevant simulator. For models 2 and 3, an emulator without derivative information requires approximately twice as many simulator runs as an emulator with derivatives to achieve similar accuracy. 5. Application to the C-GOLDSTEIN climate model Figure 13. Air temperature at the year Figure 13 shows some output of C- GOLDSTEIN run to AD 2000 under default parameters settings. emulator mean is much closer to the true function output and the uncertainty much smaller than the emulator without derivatives (shown in Figure 1). Figure 4 - Toy Model 1Figure 5 - Toy Model 2Figure 6 - Toy Model 3 2. Use of Derivatives in Emulating Model Output C-GOLDSTEIN = Coupled Global-Linear Drag Salt and Temperature Equation Integrator. It is composed of a 3-d ocean model coupled with a 2-d energy moisture balance model of the atmosphere and a simple sea ice model. C-GOLDSTEIN has simplified physics and a low resolution (36x36 grid). We apply the method of Section 4 to C-GOLDSTEIN. 30 runs of the climate model are performed, varying 3 of the input parameters. An emulator is built with the resulting output of global mean air temperature. Validation data is produced by adopting the finite differences method; 50 derivatives with respect to each input are generated.. Figure 14 (left). A comparison of the derivatives of global mean air temperature with respect to atmospheric moisture diffusivity, at the year There are some areas where the emulated derivatives match those produced by finite differences quite well. A number of points show differences between the two methods though and additional runs of the simulator are likely to be required to improve the overall performance of the emulator. The corresponding plots for the 2 remaining parameters, ocean vertical diffusivity and atmospheric heat diffusivity, are omitted here but show similar patterns.