1. Sarah Dance DARC/University of Reading
Data Assimilation Theory and Practice: Lessons learned from the Atmosphere Carbon Fusion Meeting May 2006
Sarah Dance
DARC/University of Reading

2. Outline Data assimilation for NWP Example: a choice of methods
Implementation matters!
Summary and conclusions

3. Some Data Assimilation Applications
1. Retrospective analysis -- Learn more about how the Earth works, by using models to interpret/extend different types of data (e.g., climate studies, atmospheric chemistry)
2. Diagnosis – Test and improve models (e.g., parameter estimation)
3. Forecasting – Use recent data to improve initial conditions for short-term predictions (e.g., numerical weather forecasting)
4. Real-time Control – Use continually changing estimates of system state to determine control actions (e.g. traffic flow)
Smoothing: Update states at all times with all meas.
Retrospective analysis, diagnosis t
Filtering: Update states only with previous measurements
Forecasting/ real time control t

4. Re-analysis of ozone data from MIPAS on Envisat

5. Initial conditions for NWP
Observations can provide some (not all) required information – O(106) obs assimilated every 6 hours
Observations can provide some information, but not all. Quality control.
Observation model relates state variables to observables e.g. interpolation, satellite radiances
Observation errors are usually assume unbiased, conditionally independent, spatially uncorrelated, Gaussian
Images from Chilbolton, UFAM, BBC, Met Office, EUMETSAT

6. Forecast models Global (60km)  Convective-scale (1-4km)
Dynamics is very different on these scales
Length of DA state vector O(107) variables
Want compatible initial conditions that are a solution of the NWP model: avoid
unslaved gravity modes
spin-up of hydrological cycle

7. Example: A choice of methods
4D-Var vs EnKF
Cf. Lorenc (2003) QJRMS

8. Measurement model Not all observation operators are linear.
Indeed some are not even differentiable e.g. scatterometer winds

9. 4D-Var Incremental method – uses linearization and adjoint of M and H.
Converges to nonlinear 4D-Var solution (conditions apply). (Lawless et al. 2004)
Minimum of nonlinear 4D-Var may not be unique.
Scatterometer ob, identity map model, B=1.0, R=0.1, xb =0.0, y=0.99
However, other obs may remove the ambiguity
Incremental 4D-Var is like a quasi-Newton method and hence has similar convergence results.
There are other nonlinear optimization methods, such as simulated annealing, that do not require the adjoint and TLM etc, which may be more suited in this case, but alos more computationally expensive.

10. 1D “Scatterometer” winds
Not invertible
Not differentiable
yo = 0.5
d = yo –H(x)
S’pose K has a positive sign. Then any ensemble members lying in the region x < -0.5 will be moved away from the observation in the update step.
Furthermore, the members which are furthest away from the observation in the first place will have the largest increments away from the observations, leading to filter divergence.
EnKF increments
= K d
K is scalar
d < 0
d < 0
d >0
d >0

11. EnKF with 1D scatterometer
x0 ~ N(0.4,1.0)
xt =0.5
X observations
ensemble
Perturbed obs EnKF with identity map as process model, to avoid confusion with nonlinear effects of obs update and 1d-scatterometer as observation.
40 member ensemble
True solution is 0.5
Observations plotted as x-s, but plotting is ambiguous, onloy one possible solution shown.
Clearly see that some ensemble members are moved away from the observations. E.g. member at x=-2 at beginning. K changes sign at next update.
Ultimately the filter completely ignores the observations, because the cross- covariance of the ensemble goes to zero due to cancellation of positive and negative ensemble members.
Cross- covariance =

12. Implementation matters!
Biased EnKF algorithms
Work with David Livings and Nancy Nichols

13. =

14. Results with ETKF (old formulation) and Peter Lynch’s swinging spring model
N=10, Perfect observations
Red ensemble mean
Blue ensemble std.
Error bars indicate obs std.
Ensemble statistics not consistent with the truth!

15. Bias and the EnKF
Many EnKF algorithms, can be put into a “square root” framework.
Define an ensemble perturbation matrix:
So, by definition of the ensemble mean

16. Square-root ensemble updates
The mean of the ensemble is updated separately.
Ensemble perturbations are updated as
where T is a (non-unique) square root of an update equation.
Thus, for consistency,
David discovered that not all implementations preserve this property.
We have now found nec. and suff. conditions for consistency.

17. Consequences The size of the ensemble spread will be too small
The ensemble will be biased
The size of the ensemble spread will be too small
Filter divergence is more likely to occur !
Care must be taken in algorithm choice and implementation

18. Summary and conclusions
Data assimilation is invaluable in many applications
Care must be taken in the way we apply in
Most schemes are loosely based on the Kalman filter that assumes
Linear models (M and H)
Gaussian statistics
Conditional independence between obs
Often additionally obs errors assumed uncorrelated
Need to at least stop and think about violating assumptions
Implementation (numerical analysis) matters!