1. Introduction to Adjoint Methods in Meteorology
Rolf Langland
Data Assimilation Section
Naval Research Laboratory
Monterey, CA
Including Material Provided by
Dr. Ronald M. Errico (NASA-UMBC)
JCSDA Summer Colloquium on Data Assimilation
Santa Fe, N.M., 31 July 2012

2. Adjoint Methods Introduction
What is an adjoint model
Examples of adjoint equations
Interpretation of adjoint sensitivity
Development / testing of adjoint code
Misunderstandings about adjoint methods
Please ask questions during presentation if something is not clear!

3. What is an Adjoint Model? [NWP context]
The transpose of a tangent linear version of a forecast model
It can be used to estimate the sensitivity of a forecast aspect (J) with respect to model initial conditions and parameters
Sensitivity information is useful for short-range forecasts (72 hr or less), subject to tangent-linear approximations

4. Why use adjoint models ?
Require information about sensitivity to initial conditions, but cannot construct the inverse of a numerical forecast model …
An adjoint model can provide good estimates of initial condition sensitivity, consistent with actual dynamics of nonlinear forecast model (group velocity, etc.)
Difficult or impossible to obtain this information by other methods

5. Adjoint Applications
Sensitivity of Forecast to Initial Conditions and Boundary Conditions – Key analysis errors
Observation Impact Information
Targeted Observing Guidance
Variational Data Assimilation – 4D-Var
Generation of Perturbations for Ensemble Forecasting
Optimal Perturbations and Singular Vectors

6. Adjoint models exist for:
Global weather prediction models
Regional models
Ocean models
Data assimilation procedures
Observation operators
Adjoint codes are used in one form or another at every operational NWP forecast center

7. Forecast and Analysis Procedure
Observation
(y)
Data
Assimilation
System
Analysis
(xa)
Forecast
Model
Forecast
(xf)
Background
(xb)
Adjoint of Forecast and Analysis Procedure
Observation
Sensitivity
(J/ y)
Adjoint of
the Data
Assimilation
System
Adjoint of the
Forecast Model
Tangent
Propagator
Analysis
Sensitivity
(J/ xa)
Gradient of
Cost Function
J: (J/ xf)
Background
Sensitivity
(J/ xb)
Observation Impact
<y-H(xb)> (J/ y)
What is the impact of the observations on measures of forecast error (J) ?

8. Data Assimilation Equation with K in observation space
BACKGROUND (6h) FORECAST
ANALYSIS
Post-multiplier
Solver (self-adjoint)
OBSERVATIONS
Temperature
Winds
Pressure
The analysis, Xa can be changed by perturbations of the observations (δy) or the background (δ Xb)
An adjoint can be used to quantify this sensitivity

9. Adjoint of Data Assimilation Equation
Sensitivity to Observations:
Adjoint of forecast model produces sensitivity to
Sensitivity to Background:
see Baker and Daley 2000, QJRMS

10. Adjoint Operator - Linear algebra 101
The inner (dot) product of two vectors = a scalar
If L is an m x n matrix, then L* = LT
Note: transpose is not the same as inverse !

11. TLM and Adjoint Equations
The linear operator L propagates a perturbation vector forward in time
TLM
final time initial time
L is a tangent linear version of a nonlinear model, M
perturbations of state variables
The adjoint operator LT propagates a sensitivity gradient vector backward in time – sensitivity of J to all elements of X at initial time
Adjoint
initial time final time
sensitivity gradients
LT is the transpose of L

12. Forecast Response Function (J)
J can be any differentiable function of the model state variables that comprise Xf
Examples:
Surface pressure
Temperature, wind component, specific humidity
Vorticity, divergence, enthalpy
Kinetic energy
Forecast error (f –a) or (f-a)2
Energy-weighted forecast error norm
Not valid: J=Anomaly Correlation Coefficient
𝜕(𝐴𝐶𝐶) 𝜕𝐗𝑓 = undefined

13. Estimating δJ using TLM and Adjoint
𝛿𝐽= 𝜕𝐽 𝜕𝐗𝑓 ,𝐋𝛿𝐗0 = 𝜕𝐽 𝜕𝐗𝑓 ,𝛿𝐗𝑓
𝛿𝐽= 𝛿𝐗0,𝐋T 𝜕𝐽 𝜕𝐗𝑓 = 𝛿𝐗0, 𝜕𝐽 𝜕𝐗0
Adjoint
Equivalent to a 1st-order Taylor Series
δx0 = can be any real or hypothetical perturbation 13

14. Obtain sensitivity to initial conditions using an adjoint model
The trajectory (analysis and forecast values of state variables) of the nonlinear forecast model are saved (at every time step if possible) from t=0 to t=f
The adjoint cost function (J) is defined
J  J/T, J/u, J/v, J/ps at t=f
The adjoint model is integrated backwards in time to obtain:
J/T, J/u, J/v, J/ps at t=0

15. Adjoint sensitivity example Navy COAMPS model
J = kinetic energy in lower troposphere
Animation is
𝜕𝐽 𝜕𝑢 at 700hPa, from 36hr to 0 hr

16. Forward (conventional) vs. adjoint sensitivity procedure
In a forward-in-time sensitivity procedure, we change the initial conditions (or observations) and evaluate the effect on the forecast (OSE)
In an adjoint sensitivity procedure we select the forecast aspect (J) and evaluate the sensitivity to the initial conditions (or observations)

17. Adjoint Sensitivity Analysis
Impacts vs. Sensitivities
A single impact study yields exact response measures
(J) for all forecast aspects with respect to the particular
perturbation investigated.
A single adjoint-derived sensitivity yields linearized
estimates of the particular measure (J) investigated
with respect to all possible perturbations.

18. The forecast error norm
A useful way to combine errors of wind, temperature, humidity and surface pressure into a costfunction
𝐽=Ʃ𝑖,𝑗,𝑘 0.5 𝐂u uf−ua 2+𝐂v vf−va 2+𝐂T Tf−Ta 2+𝐂q qf−qa 2+𝐂p pf−pa 2 𝐂u uf−ua 2+𝐂v vf−va 2+𝐂T Tf−Ta 2+𝐂q qf−qa 2+𝐂p pf−pa 2
𝐂u 𝐂v 𝐂T 𝐂q 𝐂p are weighting functions that transform the errors of winds, temperature, humidity and pressure into units of energy [J kg-1], accounting for grid volume size/mass …
The adjoint model starting condition is then …for example
𝜕𝐽 𝜕𝑢𝑓 = 𝐂u uf−ua , at all grid points (i,j,k), units = Jkg-1m-1s
Energy norm ref: Rabier et al. 1996, QJRMS

19. Sensitivity summary field
Then, to display of initial condition sensitivity, we can transform the gradients back into units of energy and combine the winds, temperature and pressure sensitivities
S0=Ʃ𝑖,𝑗,𝑘 𝐂 u 𝜕𝐽 𝜕𝑢0 + 𝐂 v 𝜕𝐽 𝜕𝑣0 + 𝐂 T 𝜕𝐽 𝜕𝑇0 + 𝐂 q 𝜕𝐽 𝜕𝑞0 + 𝐂 p 𝜕𝐽 𝜕p0 -1 -1 -1 -1 -1
𝐂u 𝐂v 𝐂T 𝐂q 𝐂p are inverses of the weighting functions, that transform the sensitivity gradients of winds, temperature, humidity and pressure into units of energy [J kg-1], accounting for grid volume size/mass -1 -1 -1 -1 -1

20. Sensitivity of NOGAPS 72-h forecast error to the initial T,u,v,ps fields 00 UTC 10 February 2002
J kg-1
In the extratropics, the leading SV or ASG usually have max amplitude in the lower troposphere, tilt into the prevailing westerlies, and correspond to growing baroclinic disturbances. (for these choices of J).
Not measures of analysis error, but indicate directions along which the analysis error is most likely to grow rapidly. May not be largest analysis error, but component that grows most rapidly.
It is these dynamically sensitive regions that should be most carefully observed.

21. 𝐾 𝐽𝑘𝑔 -1
𝑚𝑠 𝐽𝑘𝑔
λ units are: -1 -1
etc.

22. Optimal correction of initial 500mb temperature based on adjoint sensitivity gradient to improve 72-hr forecast of east coast storm

23. Optimal correction of initial 300mb u-wind based on adjoint sensitivity gradient to improve hr forecast of east coast storm

24. 300mb u-wind – Comparison of initial conditions -
Original IC – this produces large forecast error
“Corrected IC” – this produces small forecast error

25. L L L L NOGAPS Sea-Level Pressure Forecasts and Analyses
72-hr forecast from 12Z 22 Jan 2000
Forecast with Adjoint-based IC Correction (+72hr)
Operational Forecast (+72hr) L L L L

26. Plots of sensitivity gradient magnitude and grid resolution
Gradient of error energy with respect to Tv 24-hours earlier Units: Jkg-1K-1
1 x 1.25 degree lat-lon grid
0.5 x degree lat-lon grid
Sensitivity magnitude on this grid is larger because a perturbation of initial conditions at any grid point represents a larger area / volume
The same sensitivity gradient has smaller magnitude on this grid because of finer resolution –
The sum of sensitivity for equal areas is independent of grid resolution
Provided by R. Todling

27. Singular Vectors in 72-hr East Coast Storm Forecast
Initial Time Total Energy 12Z 22 Jan 2000
Final Time Sfc Pressure 12Z 25 Jan 2000
SV 1
SV 2
SV 3

28. Example of TLM and Adjoint Derivation
Nonlinear Equations
1st order Linearization

29. Example of TLM and Adjoint Derivation
Tangent Linear Equations
Adjoint Equations (LT)
L = the Jacobian of the nonlinear model operator
Can be derived continuous -> discrete or
discrete -> discrete
Automatic adjoint code generators
Shallow water equations

30. Nonlinear model and tangent linear model trajectories
The trajectory of the TLM is an approximation of the nonlinear trajectory
TLM
NLM
The nonlinear forecast trajectory is saved at specified time intervals and used in the TLM
The TLM is tangent to the nonlinear trajectory at every time step where an update is provided – it is not a purely linear calculation
Time

31. Development of TLM and Adjoint Model Code
OPTIONS:
Develop TLM code directly from nonlinear model code, then develop adjoint code
Develop TLM versions of nonlinear model original equations, then develop TLM and adjoint code
Option 1 is the best method ….

32. Why develop the TLM from the nonlinear model code?
Eventually a TLM and adjoint code will be necessary anyway
The code itself is the most accurate description of the model algorithm
If the model algorithm creates different dynamics than the original equations being modeled, for most applications it is the former that are desirable and only the former that can be validated

33. Automatic Differentiation Software
Input: Nonlinear code Output: TLM and adjoint code
TAMC Ralf Giering (superceded by TAF)
TAF FastOpt.com
ADIFOR Rice University
TAPENADE INRIA, Nice
OPENAD Argonne
Others

34. Considerations in development of TLM and Adjoint code
TLM and Adjoint models are straight-forward (although tedious) to derive from NLM code, and actually simpler to develop
Intelligent approximations can be made to improve efficiency
TLM and adjoint codes are simple to test rigorously
4. Some outstanding errors and problems in the NLM are typically
revealed when the TLM and Adjoint are developed
5. Some approximations to the NLM physics are generally necessary

35. TLM validation Comparison to nonlinear model
Does the TLM or Adjoint model tell us anything about
the behavior of perturbation growth in the nonlinear
model that may be of interest?

36. Linear vs. Nonlinear results with moist physics, 24-hr forecasts
TLM
NLM
Non-Convective
Precip. ci=0.5mm
Non-Convective
Precip. ci=0.5mm
TLM
NLM
Convective
Precip. ci=0.2mm
Convective
Precip. ci=0.2mm

37. Adjoint model verification Gradient check
Adjoint model starting condition and IC sensitivity gradient
𝛿𝐗𝑓, 𝜕𝐽 𝜕𝐗𝑓 = 𝛿𝐗0, 𝜕𝐽 𝜕𝐗0
Evolved TLM perturbation and TLM starting condition
Note that the dot product in the above equation is computed for all i,j,k in the model domain. This is a fundamental test to determine if the TLM and adjoint are coded correctly

38. Adjoint method accuracy
Quantitative accuracy best for small perturbations and short forecasts
Often, qualitatively useful information can be obtained for large perturbations in forecasts as long as 3-5 days…for example, when applied to mid-latitude winter storms – synoptic-scales
Accuracy is less for highly non-linear flows and smaller-scales
Adjoint accuracy is equivalent to that of the TLM

39. Tangent Linear vs. Nonlinear Results
In general, agreement between TLM and NLM results
will depend on:
Amplitude of perturbations
Stability properties of the reference state
Structure of perturbations
Physics involved
Time period over which perturbation evolves
Metric used for comparison

40. Issues with physics in TLM and adjoint
Parts of the NLM code may be non-differentiable, requiring approximations in the TLM and adjoint
2. Numerical instabilities may occur in the TLM as a result of physics linearization
3. Some physical parameterizations are much more suitable than others for linearization
4. Development of the TLM and adjoint may uncover problems with the nonlinear model physics

41. Example of a transient instability in a TLM solution
NLM
Errico and
Raeder 1999
QJRMS

42. Adjoint models as paradigm changers
Some results discovered using adjoint models
1. Atmospheric flows are very sensitive to low-level T perturbations
2. Evolution of a barotropic flow can be very sensitive to perturbations
having small vertical scale
3. Error structures can propagate and amplify rapidly
4. Forecast barotropic vorticity can be sensitive to initial water vapor
5. Relatively few perturbation structures are initially growing ones
6. Sensitivities to observations differ from sensitivities to analyses

43. Misunderstanding # 1 False: Adjoint models are difficult to understand
True: Understanding how to use and interpret adjoints of numerical models primarily uses concepts taught in early college mathematics

44. Misunderstanding # 2 False: Adjoint models are difficult to develop
True: Adjoint models of dynamical cores are simpler
to develop than their parent models, and almost
trivial to check, but adjoints of model physics
can pose difficult problems

45. Misunderstanding # 3
False: Automatic adjoint generators easily generate
perfect and useful adjoint models
True: Problems can be encountered with automatically
generated adjoint codes that are inherent in the
parent model. Do these problems also have a
bad effect in the parent model?

46. Misunderstanding # 4
False: An adjoint model is demonstrated useful and
correct if it reproduces nonlinear results for
ranges of very small perturbations
True: To be truly useful, adjoint results must yield
good approximations to sensitivities with
respect to meaningfully large perturbations.
This must be part of the validation process

47. Misunderstanding # 5
False: Adjoints are not needed because the EnKF is
better than 4DVAR and adjoint results disagree
with our notions of atmospheric behavior
True: Adjoint models have uses beyond 4DVAR. Their results can be surprising, but have been confirmed. It is rare that we have a tool that can answer such important questions so directly! It has not been demonstrated that EnKF is superior to TLM/adjoint for either data assimilation or sensitivity calculations.

48. Challenges Develop new adjoint models
Include more physics in adjoint models
Develop parameterization schemes suitable
for linearized applications
Always validate adjoint results (linearity)
Many applications (sensitivity analysis, model tuning, predictability research, etc.) for adjoint models are not being fully examined at the present time

49. Adjoint Workshops 1992 – Pacific Grove, CA 1995 – Visegrad, Hungary
1998 – Lennoxville, Quebec
2000 – Moliets-et-Maa, France
2002 – Mount Bethel, PA
2004 – Aquafredda di Maratea,Italy
2006 – Tirol, Austria
2009 – Tannersville, PA
Cefalu,Sicily
2013?
Contact Dr. Ron Errico
to be put on mailing list

50. The Energy Norm