1. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20091 DA Variational Data Assimilation Hans Huang NCAR Acknowledge: Dale Barker, The WRFDA team, MMM and DATC staff AFWA, NSF, KMA, CWB, BMB, AirDat

2. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20092 DA Introduction to data assimilation. Basics of variational data assimilation. Demonstration with a simple system. The WRFDA approach. Outline

3. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20093 DA Modern weather forecast (Bjerknes,1904) Analysis: using observations and other information, we can specify the atmospheric state at a given initial time: “Today’s Weather” Forecast: using the equations, we can calculate how this state will change over time: “Tomorrow’s Weather” Vilhelm Bjerknes (1862–1951) A sufficiently accurate knowledge of the state of the atmosphere at the initial time A sufficiently accurate knowledge of the laws according to which one state of the atmosphere develops from another. (Peter Lynch)

4. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20094 DA initial stateforecast Analysis Model Model state x, ~10 7 Observations y o, ~10 5 -10 6

5. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20095 DA

6. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20096 DA Assimilation methods Empirical methods –Successive Correction Method (SCM) –Nudging –Physical Initialisation (PI), Latent Heat Nudging (LHN) Statistical methods –Optimal Interpolation (OI) –3-Dimensional VARiational data assimilation (3DVAR) –4-Dimensional VARiational data assimilation (4DVAR) Advanced methods –Extended Kalman Filter (EKF) –Ensemble Kalman Filter (EnFK)

7. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20097 DA Introduction to data assimilation. Basics of variational data assimilation. Demonstration with a simple system. The WRFDA approach. Outline

8. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20098 DA The analysis problem for a given time

9. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 20099 DA BLUE: the Best Liner Unbiased Estimate

10. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200910 DA The analysis value should be between b ackground and observation. Statistically, analyses are better than background. Statistically, analyses are better than observations! If B is too small, observations are less u seful. If R can be tuned, analysis can fit observations as close as one wants!

11. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200911 DA 3D-Var

12. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200912 DA Sequential data assimilation (I)

13. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200913 DA Sequential data assimilation (II)

14. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200914 DA Sequential data assimilation (III)

15. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200915 DA Sequential data assimilation (IV) 4DVAR (continue)

16. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200916 DA Scalar to Vector Observation operator H

17. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200917 DA H - Observation operator H maps variables from “model space” to “observation space” x y –Interpolations from model grids to observation locations –Extrapolations using PBL schemes –Time integration using full NWP models (4D-Var in generalized form) –Transformations of model variables (u, v, T, q, p s, etc.) to “indirect” observations (e.g. satellite radiance, radar radial winds, etc.) Simple relations like PW, radial wind, refractivity, … Radar reflectivity Radiative transfer models Precipitation using simple or complex models … !!! Need H, H and H T, not H -1 !!!

18. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200918 DA Level of preprocessing and the observation operator Phase and amplitude Ionosphere corrected observables Refractivity profiles Bending angle profiles Temperature profiles Raw data: Frequency relations Geometry Abel trasform or ray tracing Hydrostatic equlibrium and equation of state H =I H =H N H =H R H N H =H G H R H N H =H F H G H R H N

19. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200919 DA Sequential data assimilation

20. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200920 DA Issues on variational data assimilation

21. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200921 DA Introduction to data assimilation. Basics of variational data assimilation. Demonstration with a simple system. The WRFDA approach. Outline

22. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200922 DA The Lorenz 1964 equation

23. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200923 DA Issues on data assimilation for the system based on the Lorenz 64 equation

24. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200924 DA Too simple? Try: Lorenz63 model Try: 1-D advection equation Try: 2-D shallow water equation … Try: ARW!!!

25. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200925 DA Introduction to data assimilation. Basics of variational data assimilation. Demonstration with a simple system. The WRFDA approach. WRFDA is a D ata A ssimilation system built within the WRF software framework, used for application in both research and operational environments…. Outline

26. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200926 DA WRF-Var (WRFDA) Data Assimilation Overview Goal: Community WRF DA system for regional/global, research/operations, and deterministic/probabilistic applications. Techniques: 3D-Var 4D-Var (regional) Ensemble DA, Hybrid Variational/Ensemble DA. Model: WRF (ARW, NMM, Global) Support: WRFDA team NCAR/ESSL/MMM/DAG NCAR/RAL/JNT/DATC Observations: Conv.+Sat.+Radar

27. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200927 DA WRFDA Observations  In-Situ: -Surface (SYNOP, METAR, SHIP, BUOY). -Upper air (TEMP, PIBAL, AIREP, ACARS, TAMDAR).  Remotely sensed retrievals: -Atmospheric Motion Vectors (geo/polar). -Ground-based GPS Total Precipitable Water. -SSM/I oceanic surface wind speed and TPW. -Scatterometer oceanic surface winds. -Wind Profiler. -Radar radial velocities and reflectivities. -Satellite temperature/humidities. -GPS refractivity (e.g. COSMIC).  Radiative Transfer: -RTTOVS (EUMETSAT). -CRTM (JCSDA). Threat Score Bias KMA Pre-operational Verification: (with/without radar)

28. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200928 DA  Assume background error covariance estimated by model perturbations x’ : Two ways of defining x’:  The NMC-method (Parrish and Derber 1992): where e.g. t2=24hr, t1=12hr forecasts…  …or ensemble perturbations (Fisher 2003):  Tuning via innovation vector statistics and/or variational methods. Model-Based Estimation of Climatological Background Errors

29. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200929 DA Single observation experiment - one way to view the structure of B The solution of 3D-Var should be Single observation

30. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200930 DA Pressure, Temperature Wind Speed, Vector, v-wind component. Example of B 3D-Var response to a single p s observation

31. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200931 DA B from ensemble methodB from NMC method Examples of B T increments : T Observation (1 Deg, 0.001 error around 850 hPa)

32. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200932 DA Sensitivity to Forecast Error Covariances in Antarctica (Rizvi et al 2006) 60km Verification (vs. radiosondes) T+24hr Temperature GFS-based errors WRF-based errors 20km Domain2 60km Domain 1

33. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200933 DA  Define preconditioned control variable v space transform where U transform CAREFULLY chosen to satisfy B o = UU T.  Choose (at least assume) control variable components with uncorrelated errors:  where n~number pieces of independent information. Incremental WRFDA J b Preconditioning

34. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200934 DA WRFDA Background Error Modeling U h : RF = Recursive Filter, e.g. Purser et al 2003 U v : Vertical correlations EOF Decomposition U p : Change of variable, impose balance.

35. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200935 DA WRFDA Background Error Modeling U h : RF = Recursive Filter, e.g. Purser et al 2003 U v : Vertical correlations EOF Decomposition U p : Change of variable, impose balance.

36. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200936 DA WRFDA Background Error Modeling U h : RF = Recursive Filter, e.g. Purser et al 2003 U v : Vertical correlations EOF Decomposition U p : Change of variable, impose balance.

37. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200937 DA WRFDA Background Error Modeling Define control variables: U h : RF = Recursive Filter, e.g. Purser et al 2003 U v : Vertical correlations EOF Decomposition U p : Change of variable, impose balance.

38. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200938 DA WRFDA Statistical Balance Constraints Define statistical balance after Wu et al (2002): How good are these balance constraints? Test on KMA global model data. Plot correlation between “Full” and balanced components of field:

39. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200939 DA WRF 4D-Var milestones 2003: WRF 4D-Var project. 2004: WRF SN (simplified nonlinear model). Modifications to WRF 3D-Var. 2005: TL and AD of WRF dynamics. WRF TL and AD framework. WRF 4D-Var framework. 2006: The WRF 4D-Var prototype. Single ob and real data experiments. Parallelization of WRF TL and AD. Simple physics TL and AD. JcDF 2007: The WRF 4D-Var basic system. 2008: Further optimization and testing

40. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200940 DA Basic system: 3 exes, disk I/O, parallel, full dyn, simple phys, JcDF NL(1),…,NL(K) TL(1),…,TL(K) AF(K),…,AF(1) AD00 WRF WRF_NL WRF_TL WRF_AD VAR Outerloop M k d k Innerloop U U T call TL00 Ry1…yKRy1…yK xbxb I/O WRFBDY B WRF+ BS(0),…,BS(N) xnxn TLDF WRFINPUT

41. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200941 DA Why 4D-Var? Use observations over a time interval, which suits most asynoptic data and use tendency information from observations. Use a forecast model as a constraint, which enhances the dynamic balance of the analysis. Implicitly use flow-dependent background errors, which ensures the analysis quality for fast developing weather systems. NOT easy to build and maintain!

42. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200942 DA Radiance Observation Forcing at 7 data slots

43. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200943 DA Single observation experiment The idea behind single ob tests: The solution of 3D-Var should be Single observation 3D-Var  4D-Var: H  HM; H  HM; H T  M T H T The solution of 4D-Var should be Single observation, solution at observation time

44. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200944 DA Analysis increments of 500mb  from 3D-Var at 00h and from 4D-Var at 06h due to a 500mb T observation at 06h ++ 3D-Var4D-Var

45. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200945 DA 500mb  increments at 00,01,02,03,04,05,06h to a 500mb T ob at 06h OBS+

46. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200946 DA 500mb  difference at 00,01,02,03,04,05,06h from two nonlinear runs (one from background; one from 4D-Var) OBS+

47. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200947 DA 500mb  difference at 00,01,02,03,04,05,06h from two nonlinear runs (one from background; one from FGAT) OBS+

48. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200948 DA Real Case: Typhoon Haitang Experimental Design (Cold-Start) Domain configuration: 91x73x17, 45km Period: 00 UTC 16 July - 00 UTC 18 July, 2005 Observations from Taiwan CWB operational database. 5 experiments are conducted:  FGS – forecast from the background [The background fields are 6-h WRF forecasts from National Center for Environment Prediction (NCEP) GFS analysis.]  AVN- forecast from the NCEP AVN analysis  3DVAR – forecast from WRF-Var3d using FGS as background  FGAT - forecast from WRF-Var3dFGAT using FGS as background  4DVAR – forecast from WRF-Var4d using FGS as background

49. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200949 DA Typhoon Haitang 2005

50. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200950 DA A KMA Heavy Rain Case Period: 12 UTC 4 May - 00 UTC 7 May, 2006 Assimilation window: 6 hours Cycling (6h forecast from previous cycle as background for analysis) All KMA operational data Grid : 60x54x31 Resolution : 30km Domain size: the same as the KMA operational 10km domain.

51. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200951 DA Precipitation Verification

52. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200952 DA Test domain: AFWA T8 45-km Domain configurations: 2007-08-15-00 to 2007-08-21-00 Horizontal grid: 123 x 111 @ 45 km grid spacing, 27 vertical levels 3/6 hours GFS forecast as FG Every 6 hours Observations used: soundsonde_sfc synopgeoamv aireppilot ships qscat gpspw gpsref

53. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200953 DA 00 h 12 h 24 h RMSE Profiles at Verification Times (U,V)

54. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200954 DA 00 h 12 h 24 h RMSE Profiles at Verification Times (T,Q)

55. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200955 DA The first radar data assimilation experiment using WRF 4D-Var (OSSE) TRUTH ----- Initial condition from TRUTH (13-h forecast initialized at 2002061212Z from AWIPS 3-h analysis) run cutted by ndown, boundary condition from NCEP GFS data. NODA ----- Both initial condition and boundary condition from NCEP GFS data. 3DVAR ----- 3DVAR analysis at 2002061301Z used as the initial condition, and boundary condition from NCEP GFS. Only Radar radial velocity at 2002061301Z assimilated (total # of data points = 65,195). 4DVAR ----- 4DVAR analysis at 2002061301Z used as initial condition, and boundary condition from NCEP GFS. The radar radial velocity at 4 times: 200206130100, 05, 10, and 15, are assimilated (total # of data points = 262,445).

56. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200956 DA TRUTHNODA4DVAR3DVAR Hourly precipitation ending at 03-h forecast

57. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200957 DA TRUTH NODA3DVAR4DVAR Hourly precipitation ending at 06-h forecast

58. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200958 DA Real data experiments

59. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200959 DA Introduction to data assimilation. Basics of variational data assimilation. Demonstration with a simple system. The WRFDA approach: y, H, H, H T B M, M, M T 4D-Var Summary

60. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200960 DA A short list of variational data assimilation issues

61. Hans Huang: Variational Data Assimilation, SNU lecture. June 9th, 200961 DA www.mmm.ucar.edu/wrf/users/wrfda