2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 1Introduction to data assimilation An introduction to data assimilation Xiang-Yu Huang Danish Meteorological Institute, Denmark

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 2Introduction to data assimilation Outline of the presentation Operational NWP activities Observations and preprocessing – There are still many observations we are not able to assimilate. – We have to prepare for new observations to come. Observation operators H Error covariances B and R – They determine the assimilation quality. – We can only guess what they should be. Data impact – It can take decades of hard work just to assimilate one data type. – How to assess data impact is application dependent. Summary and our near future plan.

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 3Introduction to data assimilation

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 4Introduction to data assimilation Numerical Weather Prediction: models and initial values

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 5Introduction to data assimilation DMI-HIRLAM The operational system consists of three nested models named "G", "E" and "D".

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 6Introduction to data assimilation Data assimilation cycles

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 7Introduction to data assimilation SYNOP SHIP BUOY

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 8Introduction to data assimilation AIREP AMDAR ACARS

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 9Introduction to data assimilation TEMP PILOT

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 10Introduction to data assimilation ATOVS

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 11Introduction to data assimilation Comments (I): Observations alone are not enough. Observations only cover part of the model domain (for limited area models they could also be outside of the model domain). Some observations provide incomplete model state at given locations (e.g. only wind). Some observations are not NWP model variables (e.g. radiance). NWP is not the only purpose of making observations.

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 12Introduction to data assimilation Quality control Observing systems have problems. Bad reporting practice check Blacklist check Gross check (against some limits) Background (short-range forecasts) check “Buddy check” (against nearby observations) Redundancy check Analysis check: OI check or VarQC

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 13Introduction to data assimilation

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 14Introduction to data assimilation

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 15Introduction to data assimilation Received Assimilated

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 16Introduction to data assimilation Comments (II): We are far from using all the observations. Data quality dependent. Observing system dependent. NWP model (resolution) dependent. Assimilation method dependent. At the same time, we have to prepare for the new data like RO to come.

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 17Introduction to data assimilation Routine monitoring Short-range forecasts - observations

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 18Introduction to data assimilation Analysis methods Empirical methods –Successive Correction Method (SCM) –Nudging –Physical Initialisation (PI), Latent Heat Nudging (LHN) Statistical methods –Optimal Interpolation (OI) –3-Dimensional VARiational data assimilation (3DVAR) –4-Dimensional VARiational data assimilation (4DVAR) Advanced methods –Extended Kalman Filter (EKF) –Ensemble Kalman Filter (EnFK)

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 19Introduction to data assimilation Variational methods (old forecast) (new) (initial condition for NWP) xx

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 20Introduction to data assimilation Algorithms

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 21Introduction to data assimilation Important issues H observation operator, including the tangent linear operator H and the adjoint operator H T. M forecast model, including the tangent linear model M and adjoint model M T. B background error covariance (NxN matrix). R observation error covariance which includes the representative error (MxM matrix).

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 22Introduction to data assimilation Observation operator H: from model state x to observations y This is mainly for conventional “point” observations. Horizontal and vertical integration (not interpolation) may be needed for most remote sensing data.

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 23Introduction to data assimilation Examples of specific observation operators For direct model variable observations, H spec = I. Radial winds: Integrated water vapour: Refractivity: For radiance data, RTTOV-7 (a complicated software).

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 24Introduction to data assimilation Level of preprocessing and the observation operator Phase and amplitude Ionosphere corrected observables Refractivity profiles Bending angle profiles Temperature profiles Raw data: Frequency relations Geometry Abel trasform or ray tracing Hydrostatic equlibrium and equation of state H spec =I H spec =H N H spec =H R H N H spec =H G H R H N H spec =H F H G H R H N

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 25Introduction to data assimilation Basic assumptions Observations are unbiased. (Bias removed.) Background is unbiased. (Bias removed?) Observation error covariance matrix is known. R Background error covariance matrix is known. B Observation errors and background errors are not correlated.

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 26Introduction to data assimilation Observation errors, computed for GPS/MET geopotential data (using ECMWF analyses as “TRUTH”)

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 27Introduction to data assimilation Estimate B without “TRUTH” The NMC method –Background error covariances are proportional to correlations of differences between 48 h and 24 h forecasts valid at the same time. The analysis ensemble method –Several analyses are performed with perturbed observations. Differences between background fields are used to estimate background error covariances.

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 28Introduction to data assimilation The Hollingsworth-Lönnberg method. (Estimate both B and R without “TRUTH”) B: R:

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 29Introduction to data assimilation ZZ VZ UZ Horizontal multivariate correlation: spread the information ZV VV ZU VU UVUU

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 30Introduction to data assimilation Wave number Pressure (hPa) Vertical correlation (spread the information) for the temperature at 500 hPa

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 31Introduction to data assimilation Analysis increments due to 5 GPS/MET Z profiles

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 32Introduction to data assimilation Comments (III) We need to estimate observation errors now and then. Observation errors include representative errors. Observation errors should be estimated for each model system. Observation errors may need to be re-estimated for each model refinement and instrument improvement. (It is believed that it is more important to get  o /  b right than to estimate  o and  b.)

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 33Introduction to data assimilation Comments (IV) We need to estimate the background errors again and again. Spread information (but could also cause “problems”) –horizontally –vertically –to other variables Impose balances to the analysis. Background errors should be estimated for each model system and be re-estimated for each model improvement. (It is believed that it is more important to get  o /  b right than to estimate  o and  b.)

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 34Introduction to data assimilation From research to operations Development and simple checks –Coding –Analysis increments Case studies Extensive experiments (e.g. one month for each season) –“Standard scores”: bias, rms, correlation, etc. –Special scores: precipitation, surface fluxes, etc. –Special aspects: noise, spin-up, etc. Pre-operational tests Operational use (feedback to further research)

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 35Introduction to data assimilation Observation verification against EWGLAM station list Jan 2003 NOA (No ATOVS) WIA (With ATOVS) MSLP T02M V10M V850 V500 V200 T850 T500 T200 Z850 Z500 Z200 RH500 RH850 ATOVS into DMI OPR since (A) TOVS work started in 1988 (Gustafsson and Svensson)

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 36Introduction to data assimilation Observed and predicted (+12h) precipitation ObservedWithout ZTDWith ZTD

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 37Introduction to data assimilation Recent HIRLAM impact studies 1. EWP: minor positive impact; blacklisting and bias correction may be needed. 2. MODIS wind: slightly negative; obs errors, screening procedures and level assignment need to be investigated. 3. MODIS IWV: neutral obsver, but positive on heavy precip cases. 4. GPS ZTD: neutral impact on most meteorological parameters, but positive impact on heavy precipitation cases. 5. AMSU-A: positive impact for the recent two-month experiment. The firstguess check is important. 6. Quikscat: positive impact

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 38Introduction to data assimilation Comments (V) We need to assess data impact regularly. It can take years and decades for an observing system to reach the operational status. An observing system in operational use may also become redundant due to advances in assimilation techniques, new observing systems and improvements in other components. Continuous monitoring and further tuning are necessary to keep an observing system in the operational use.

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 39Introduction to data assimilation Other important aspects Balanced motion Adjustment and initialisation Flow dependent B Non-Gaussian statistics

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 40Introduction to data assimilation Summary Observations alone are not enough. We are far from using all the available observations, and at the same time we have to prepare for the new data to come. The “statistics” is evolving: –Observational errors –Background errors It is getting more difficult for a new observing system to have a positive impact, as –NWP models become better –Other existing observing systems become better

2nd GRAS SAF User Workshop, June 2003, Helsingør, Denmark. 41Introduction to data assimilation Assimilating Radio Occultation data Global data coverage. Good vertical resolution (in contrast to most other satellite data). Insensitive to cloud and precipitation. Positive impact from real data collected from a single LEO has already been found on one of the most advanced data assimilation systems. We will start soon after this workshop - next week!