Presentation on theme: "Tangent linear and adjoint models for variational data assimilation"— Presentation transcript:
1 Tangent linear and adjoint models for variational data assimilation Angela Benedettiwith contributions from:Marta Janisková , Yannick Tremolet, Philippe Lopez,Lars Isaksen, and Gabor Radnoti
2 Introduction4D-Var is based on minimization of a cost function which measures the distance between the model with respect to the observations and with respect to the background stateThe cost function and its gradient are needed in the minimization.The tangent linear model provides a computationally efficient (although approximate) way to calculate the model trajectory, and from it the cost function. The adjoint model is a very efficient tool to compute the gradient of the cost function.Overview:Introduction to 4D-VarGeneral definitions of Tangent Linear and Adjoint models and why they are extremely useful in variational assimilationWriting TL and AD modelsTesting themAutomatic differentiation software (more on this in the afternoon)
3 4D-Var In 4D-Var the cost function can be expressed as follows: B background error covariance matrix,R observation error covariance matrix (instrumental + interpolation +observation operator error),H observation operator (model space observation space),M forward nonlinear forecast model (time evolution of the model state).H’T = adjoint of observation operator and M’T = adjoint of forecast model.
4 Incremental 4D-Var at ECMWF In incremental 4D-Var, the cost function is minimized in terms of increments:with the model state defined at any time ti as:is the trajectory around which the linearization is performed (at t=0)4D-Var cost function can then be approximated to the first order by:whereis the so-called departure.The gradient of the cost function to be minimized is:and are the tangent linear models which are used in the computations of incremental updates during the minimization (iterative procedure).and are the adjoint models which are used to obtain the gradient of the cost function with respect to the initial condition.
5 Details on linearisation In the first order approximation, a perturbation of the control variable (initial condition) evolves according to the tangent linear model:where i is the time-step.The perturbation of the cost function around the initial state is:where is the linearised version of about andare the departures from observations.
6 Details of the linearisation (cnt.) The gradient of the cost function with respect to is given by:remembering thatThe optimal initial perturbation is obtained by finding the valueof for which:The gradient of the cost function with respect to the initial condition is provided by the adjoint solution at time t=0.
7 Definition of adjoint operator For any linear operator there exist an adjoint operatorsuch as:where is an inner scalar product and x, y are vectors(or functions) of the space where this product is defined.It can be shown that for the inner product defined in theEuclidean space :We will now show that the gradient of the cost function at timet=0 is provided by the solution of the adjoint equations at thesame time:
8 Adjoint solutionUsually the initial guess is chosen to be equal to thebackground so that the initial perturbationThe gradient of the cost function is hence simplified as:We choose the solution of the adjoint system as follows:We then substitute progressively the solution into theexpression for
9 Adjoint solution (cnt.) Finally, regrouping and remembering that and thatand we obtain the following equality:The gradient of the cost function with respect to the control variable (initial condition) is obtained by a backward integration of the adjoint model.
10 Iterative steps in the 4D-Var Algorithm Integrate forward model gives .Integrate adjoint model backwards givesIf then stop.Compute descent direction (Newton, CG, …).Compute step size :Update initial condition:
11 Finding the minimum of cost function J iterative minimization procedurecost function JJ(xb)Jminimodel variable x2model variable x1
12 2 minimizations in the old An analysis cycle in 4D-Var1st ifstraj:Non-linear model is used to compute the high-restrajectory (T1279 operational, 12-h forecast)High-res departures are computed at exact obstime and locationTrajectory is interpolated at low res (T159)1st ifsmin (70 iterations):Iterative minimization at T159 resolutionTangent linear with simplified physics to calculatethe incrementsThe Adjoint is used to compute the gradient of thecost function with respect to the departure ininitial conditionAnalysis increment at initial time is interpolatedback linearly from low-res to high-res and it providesa new initial state for the 2nd trajectory run2nd ifstraj:repeat 1st ifstraj and interpolates at T255 resolution2nd ifsmin (30 iterations):repeat 1st ifsmin at T255Last ifstraj:Uses updated initial condition to run another 12-hforecast and stores analysis departures in theObservational Data Base (ODB)2 minimizations in the oldconfigurationNow 3 minimizationsare operational!
15 Simple example of adjoint writing (cnt.) (often the adjoint variables are indicated in literature with an asterisk)As an alternative to the matrix method, adjoint coding can be carried out using a line-by-line approach.
16 More practical examples on adjont coding: the Lorenz model where is the time, the Prandtl number, the Rayleigh number, the aspect ratio, the intensity of convection, the maximum temperature difference and the stratification change due to convection (see references).
17 The linear code in Fortran Linearize each line of the code one by one:dxdt(1) =-p*x(1) +p*x(2) :Nonlinear statement(1)dxdt_tl(1)=-p*x_tl(1)+p*x_tl(2) :Tangent lineardxdt(2) = x(1)*(r-x(3)) -x(2)(2)dxdt_tl(2)=x_tl(1)*(r-x(3))-x(1)*x_tl(3) -x_tl(2)…etcIf we drop the _tl subscripts and replace the trajectory x, with x5 (as it is per convention in the ECMWF code), the tangent linear equations become:(1) dxdt(1)=-p*x(1)+p*x(2)(2) dxdt(2)=x(1)*(r-x5(3))-x5(1)*x(3)-x(2)…Similarly, the adjoint variables in the IFS are indicated without any subscripts(it saves time when writing tangent linear and adjoint codes).
18 Trajectory The trajectory has to be available. It can be: saved which costs memory,recomputed which costs CPU time.Intermediate options exist using check-pointing methods.
19 Adjoint of one instruction From the tangent linear code: dxdt(1)=-p*x(1)+p*x(2)In matrix form, it can be written as:which can easily be transposed:The corresponding code is:x(1)=x(1)-p*dxdt(1)x(2)=x(2)+p*dxdt(1)dxdt(1)=0
20 The Adjoint Code Property of adjoints (transposition): Application: where represents the line of the tangent linear model.The adjoint code is made of the transpose of each line of the tangent linear code in reverse order.
21 Adjoint of loops In the TL code for the Lorenz model we have: DO i=1,3 x(i)=x(i)+dt*dxdt(i)ENDDOwhich is equivalent tox(1)=x(1)+dt*dxdt(1)x(2)=x(2)+dt*dxdt(2)x(3)=x(3)+dt*dxdt(3)We can transpose and reverse the lines:dxdt(3)=dxdt(3)+dt*x(3)dxdt(2)=dxdt(2)+dt*x(2)dxdt(1)=dxdt(1)+dt*x(1)DO i=3,1,-1dxdt(i)=dxdt(i)+dt*x(i)
22 Conditional statements What we want is the adjoint of the statements which were actually executed in the direct model.We need to know which branch was executedThe result of the conditional statement has to be stored: it is part of the trajectory !!!
23 And do not forget to initialize local adjoint variables to zero ! Summary of basic rules for line-by-line adjoint coding (1)Adjoint statements are derived from tangent linear ones in a reversed orderTangent linear codeAdjoint codeδx = 0δx* = 0δx = A δy + B δzδy* = δy* + A δx*δz* = δz* + B δx*δx = A δx + B δzδx* = A δx*do k = 1, Nδx(k) = A δx(k1) + B δy(k)end dodo k = N, 1, 1 (Reverse the loop!)δx*(k 1) = δx*(k1) + A δx*(k)δy*(k ) = δy*(k) + B δx*(k)δx*(k) = 0if (condition) tangent linear codeif (condition) adjoint codeOrder of operations is importantwhen variable is updated!And do not forget to initialize local adjoint variables to zero !
24 Trajectory and adjoint code Summary of basic rules for line-by-line adjoint coding (2)To save memory, the trajectory can be recomputed just before the adjoint calculations.Tangent linear codeTrajectory and adjoint codeif (x > x0) thenδx = A δx / xx = A Log(x)end ifTrajectoryxstore = x (storage for use in adjoint)Adjointif (xstore > x0) thenδx* = A δx* / xstoreThe most common sources of error in adjoint coding are:Pure coding errorsForgotten initialization of local adjoint variables to zeroMismatching trajectories in tangent linear and adjoint (even slightly)Bad identification of trajectory updates
25 More remarks about adjoints The adjoint always exists and it is unique, assuming spaces of finite dimension. Hence, coding the adjoint does not raise questions about its existence, only questions of technical implementation.In the meteorological literature, the term adjoint is often improperly used to denote the adjoint of the tangent linear of a non-linear operator. In reality, the adjoint can be defined for any linear operator. One must be aware that discussions about the existence of the adjoint usually address the existence of the tangent linear model.Without re-computation, the cost of the TL is usually about 1.5 times that of the non-linear code, the cost of the adjoint between 2 and 3 times.The tangent linear model is not strictly necessary to run 4D-Var (but it is in the incremental 4D-Var formulation in use operationally at ECMWF). It is also needed as an intermediate step to write the adjoint.
26 Test for tangent linear model machine precision reachedPerturbation scaling factor
27 Test for adjoint modelThe adjoint test is truly unforgiving. If you do not have a ratio of thenorm close to 1 within the precision of the machine, you know there is a bugin your adjoint. At the end of your debugging you will have a perfect adjoint(although you may still have an imperfect tangent linear!)
28 Test of adjoint in practice… (example from the aerosol assimilation) Compute perturbed variable (for example optical depth, ) using perturbation in input variables (for example, mixing ratio, , and humidity, ) with the tangent linear codeCall adjoint routine to obtain gradients in and with respect to initial condition ( and ) from perturbation inCompute the norm from the adjoint calculation, using unperturbed state and gradients:According to the test of adjoint NORM_TL must be equal to NORM_AD to the machine precision!
29 Automatic differentiation Because of the strict rules of tangent linear and adjoint coding,automatic differentiation is possible.Existing tools: TAF (TAMC), TAPENADE (Odyssée), ...Reverse the order of instructions,Transpose instructions instantly without typos !!!Especially good in deriving tangent linear codes!There are still unresolved issues:It is NOT a black box tool,Cannot handle non-differentiable instructions (TL is wrong),Can create huge arrays to store the trajectory,The codes often need to be cleaned-up and optimised.
30 Useful References Variational data assimilation: Lorenc, A., 1986, Quarterly Journal of the Royal Meteorological Society, 112,Courtier, P. et al., 1994, Quarterly Journal of the Royal Meteorological Society, 120,Rabier, F. et al., 2000, Quarterly Journal of the Royal Meteorological Society, 126,The adjoint technique:Errico, R.M., 1997, Bulletin of the American Meteorological Society, 78,Tangent-linear approximation:Errico, R.M. et al., 1993, Tellus, 45A,Errico, R.M., and K. Reader, 1999, Quarterly Journal of the Royal Meteorological Society, 125,Janisková, M. et al., 1999, Monthly Weather Review, 127,Mahfouf, J.-F., 1999, Tellus, 51A,Lorenz model:X. Y. Huang and X. Yang. Variational data assimilation with the Lorenz model. Technical Report 26, HIRLAM,April 1996.E. Lorenz. Deterministic nonperiodic flow. J. Atmos. Sci., 20: , 1963.Automatic differentiation:Giering R., Tangent Linear and Adjoint Model Compiler, Users Manual Center for Global ChangeSciences, Department of Earth, Atmospheric, and PlanetaryScience,MIT,1997Giering R. and T. Kaminski, Recipes for Adjoint Code Construction, ACM Transactions onMathematical Software, 1998TAMC:TAPENADE:Sensitivity studies using the adjoint techniqueJaniskova, M. and J.-J. Morcrette., Investigation of the sensitivity of the ECMWF radiation scheme to input parameters using adjoint technique. Quart. J. Roy. Meteor. Soc., 131,