A Variational Carbon Data Assimilation System (CDAS-4DVar) David Baker & David Schimel NCAR / Terrestrial Sciences Section, Scott Doney Woods Hole Oceanographic Institution 19 Sept 2006 We’ve heard a lot over the past week about different assimilation methods and their details. It might seem a bit overwhelming to understand how all of these methods relate to each other, and what their various strengths and weaknesses are. In this second week of the workshop, my job is to give you an example problem and show you how these different methods would be applied to it. Oftentimes, the theoretical aspects of a method become much clearer when confronted with real-world problems, as we will do here.

Main Points A 4D-Var assimilation system to compute time-varying surface fluxes, based around the PCTM transport model driven by GEOS-DAS3 fields Extensively tested using simulated data -- works!! Used to do OSSEs for OCO satellite and other potential observing networks Method can solve for other control variables besides fluxes if models available Caveat: a true adjoint to the full (non-linear) advection scheme is still needed Main conclusions given up front…

Outline 4DVar assimilation method 4DVar vs. EnKF: pros & cons PCTM transport model Mathematical details Perfect-model experiments to test it works to assess the impact of data errors to assess the impact of the prior OSSE Experiments current and future in situ networks OCO satellite dense surface coverage 4DVar vs. EnKF: pros & cons

Atmospheric Transport Model Parameterized Chemical Transport Model (PCTM; Kawa, et al, 2005) Driven by reanalyzed met fields from NASA/Goddard’s old GEOS-DAS 3 scheme Lin-Rood finite volume advection scheme Vertical mixing: diffusion plus a simple cloud convection scheme Exact adjoint for linear advection case Basic resolution 2°x 2.5°, 25  layers, t »30 min, with ability to reduce resolution to 4°x 5°, t »60 min 6°x 10°, t »120 min <-- we’ll use this in the example 12°x 15°, t »180 min Measurements binned at t resolution Adjoint coded manually -- runs as quickly as forward model The 4DVar method is built around the FWD and ADJ versions of an offline ransport model, PCTM.

4-D Var Data Assimilation Method (Baker, et al, 2006) Find optimal fluxes u and initial CO2 field xo to minimize subject to the dynamical constraint where x are state variables (CO2 concentrations), h is a vector of measurement operators z are the observations, R is the covariance matrix for z, uo is an a priori estimate of the fluxes, Puo is the covariance matrix for uo, xo is an a priori estimate of the initial concentrations, Pxo is the covariance matrix for xo, is the transition matrix (representing the transport model), and G distributes the current fluxes into the bottom layer of the model Note change in notation: fluxes are now denoted u, and concentrations x. The delta in the measurement-mismatch term implies a summation over all measurements j that fall within flux timespan i -- if flux at time j falls within i, then ij =1, otherwise it =0.

4-D Var Data Assimilation Method (Baker, et al, 2006) Adjoin the dynamical constraints to the cost function using Lagrange multipliers  Setting F/xi = 0 gives an equation for i, the adjoint of xi: The adjoints to the control variables are given by F/ui and F/xo as: The gradient vector is given by F/ui and F/xoo, so the optimal u and x0 may then be found using one’s favorite descent method. I have been using the BFGS method, since it conveniently gives an estimate of the leading terms in the covariance matrix. Note that there are no matrixes to store (other than R-1 and P-1, which can be computed once) or invert, and no basis functions H to pre-compute.

4-D Var Iterative Optimization Procedure 0 x0 ° Estimated Fluxes Forward Transport Forward Transport “True” Fluxes 1 2 Modeled Concentrations “True” Concentrations ° x1 ° Measurement Sampling Measurement Sampling x2 Modeled Measurements “True” Measurements Assumed Measurement Errors D/(Error)2 3 x3 ° Weighted Measurement Residuals Figure showing minimization approach with the 4-D Var method. The minimum is found by iteratively descending down the manifold, first doing a FWD run to set the measurements and obtain the cost function value, then doing an ADJ run to get the gradient vector, then doing a 1-D search in the down-gradient direction to find the local minimum, and repeating. Flux Update Adjoint Transport Adjoint Fluxes =  Minimum of cost function J

Perfect-model Experiments 2-hourly measurements in the lowest model level at 6°x 10°, 1 ppm error (1) Iterate 30 descent steps, 1-year-long run, starts Jan 1st 4 cases Case 1 -- No measurement noise added, no prior Case 2 -- Add measurement noise added, no prior Case 3 -- Add noise, and apply a prior Case 4 -- No noise, but apply a prior Designed to test the method and understand the impact of data errors and the usefulness of the prior Case 3 is the most realistic and will be used to do OSSEs for several possible future networks

Assimilation results using dense surface measurements Prior – True Flux |Prior – Truth| · 10-8 [ kg CO2 m-2 s-1 ] Assimilation results using dense surface measurements Estimated - True Flux |Est.-Truth|-|Prior-Truth| No data errors No prior constraint W/ data errors No prior constraint W/ data errors W/ prior constraint No data errors W/ prior constraint

Experiment #1: Convergence of the flux error Iteration 1.0 60 Relative |Estimate-Truth| Current in situ Extended in situ OCO-column OCO-surface Dense-column Dense-surface No data error, no prior With data error, no prior With data error, with prior No data error, with prior Experiment #1: Convergence of the flux error

OSSE Experiments Use Case 3 from above to test more-realistic measurement networks: Current in situ network Extended version of current network OCO satellite 2-hourly 6° x 10° column measurements 2-hourly 6° x 10° in situ measurements (from the perfect-model experiments above) An “OSSE” (observing system simulation experiment) tells you how well your measurements should constrain the fluxes. Only random error part, not biases

The five networks tested in Experiment #2 “Current in situ” – the current in situ CO2 network, with ~100 weekly flask sites and ~30 continuous sites. “Extended in situ” – an extended version of the current in situ network, with continuous analyzers placed around the globe in places with wide footprints (on tall towers or aircraft over land, or in the middle of the ocean). Variable precision, 0.5-20 ppmv (1s). ~180 measurements/hour. “OCO” – simulated measurements from the polar-orbiting OCO satellite. 16 north-south day-time ground tracks per day. Column-integrated concentrations with a 1 ppmv (1s) precision. ~70 measurements/hour. “Dense-column” – every horizontal grid location gets one column-integrated measurement per hour, at 1 ppmv (1s) precision. “Dense-surface” – as in 4., but with measurement taken in the bottom model layer.

Dense coverage, 6°x 10°, 2-hourly

Across 1 day OCO Groundtrack, Jan 1st (Boxes at 6x 10) 2 days 5 days

Extended Surface Network 3 networks, 3 OSSEs

OSSE Results For Five CO2 Measurement Networks · 10-8 [ kg CO2 m-2 s-1 ] OSSE Results For Five CO2 Measurement Networks

For retrospective analyses, a 2-sided smoother gives more accurate estimates than a 1-sided filter. The 4-D Var method is 2-sided, like a smoother. Three types of smoothers: fixed lag, fixed point, fixed interval. Since we have multiple times in our state, we effectively have a fixed-lag Kalman smoother. Fixed point and fixed interval require an adjoint, fixed lag doesn’t. Thus, one of the key advantages of the EnKF (that it doesn’t require an adjoint) applies to only those near-real-time problems for which fixed-lag smoothers work well. For true retrospective work, it helps to have an adjoint, and then EnKF vs. 4DVar comes down to a debate over run-time vs. storage and over parallelization considerations. (Gelb, 1974)

Why use a variational approach? 4DVar vs. ensemble Kalman filter (EnKF) 4DVar requires an adjoint model to back-propagate information -- this can be a royal pain to develop! The EnKF can get around needing an adjoint by using a filter-lag rather than a fixed-interval Kalman smoother. However, the need to propagate multiple time steps in the state makes it less efficient than the 4DVar method Both give a low-rank estimate of the a posteriori covariance matrix Both can account for dynamic errors Both calculate time-evolving correlations between the state and the measurements The choice between going with the fixed-interval versus fixed-lag smoother boils down to a tradeoff between the additional run-time cost of the fixed-lag smoother versus the up-front cost of developing an adjoing for the fixed-interval smoother.

References Baker, D., Doney, S., and Schimel, D., Variational Data Assimilation for Atmospheric CO2, Tellus-B, 2006 (in press) Gelb, A., et al., Applied Optimal Estimation, The M.I.T. Press, 1974, 374 pp. Kawa, S.R., Erickson, D.J., Pawson, S., and Zhu, Z., Global CO2 transport simulations using meteorological data from the NASA data assimilation system, J. Geophys. Res., 109 (D18): D18312, Sep 29, 2004.