1. Implementation plans of Hybrid 4D EnVar for the NCEP GFS
Rahul Mahajan1,4, Daryl Kleist1,3, Jeff Whitaker2, and John Derber1
1NOAA/NWS/NCEP/EMC
2NOAA/OAR/ESRL/PSD
3University of Maryland
4IMSG
With contributions from Lili Lei (CIRES), Misha Rancic (EMC), Catherine Thomas (IMSG) and many others
WWOSC 2014, Montreal

2. Ensemble-Var methods: nomenclature
En-4DVar: Propagate ensemble Pb from one assimilation window to the next (updated using EnKF for example), replace static Pb with ensemble estimate of Pb at start of 4DVar window, Pb propagated with tangent linear model within window.
4D-EnVar: Pb at every time in the assim. window comes from ensemble estimate (TLM no longer used).
As above, with hybrid in name: Pb is a linear combination of static and ensemble components.
3D-EnVar: same as 4D ensemble Var, but Pb is assumed to be constant through the assim. window (current NCEP implementation).

3. CURRENT Dual-Res Coupled Hybrid Var/EnKF Cycling
Generate new ensemble perturbations given the latest set of observations and first-guess ensemble
member 1
forecast
member 1
analysis
T254L64
EnKF
member update
member 2
forecast
member 2
analysis
recenter analysis ensemble
member 3
analysis
member 3
forecast
Ensemble contribution to background error covariance
Replace the EnKF ensemble mean analysis
and inflate
GSI 3D
Hybrid Ens/Var
high res
forecast
high res
analysis
T574L64
Previous Cycle
Current Update Cycle

4. 4D EnVar: The Way Forward ?
Natural extension to operational 3D EnVar
Uses variational approach with already available 4D ensemble perturbations
No need for development of maintenance of TLM and ADJ models
Makes use of 4D ensemble to perform 4D analysis
Modular, usable across a wide variety of models
Highly scalable
Aligns with technological/computing advances
Computationally inexpensive relative to 4DVAR (with TL/AD)
Estimates of improved efficiency by 10x or more, e.g. at Env. Canada (6x faster than 4DVAR on half as many cpus)
Compromises to gain best aspects of (4D) variational and ensemble DA algorithms
Other centers exploring similar path forward for deterministic NWP
Canada (potentially replace 4DVAR), UKMO (potentially replace En4DVar)

5. Single Observation (-3h) Example for 4D Variants
TLMADJ
4DENSV
ENSONLY
H-4DVAR_AD
bf-1=0.25
H-4DENSV
bf-1=0.25
TLMADJ

6. Low Resolution GFS/GDAS Experiments with real observations
Basic configuration
T254L64 GFS, opnl obs, GFS/GDAS cycles
PR3LOEX0
3DVAR
PRHLOEX1
Hybrid 3D EnVar, 80 member T126L64 ensemble with fully coupled (two-way) EnKF update, slightly re-tuned localization and inflation for lower resolution, TLNMC on total increment, 75% ensemble & 25% static
PRH4DEX1
Hybrid 4D EnVar, TLNMC on all time levels, only 1x150 iterations
Hourly TC relocation, O-G, binning of observations

7. 500 hPa Die Off Curves
Southern Hemisphere
Northern Hemisphere
4DHYB ----
3DHYB ----
3DVAR ----
4DHYB ----
3DHYB ----
3DVAR ----
4DHYB-3DHYB
3DVAR-3DHYB
Move from 3D Hybrid (current operations) to Hybrid 4D-EnVar yields improvement that is about 75% in amplitude in comparison from going to 3D Hybrid from 3DVAR.

8. (Preliminary) Results and Comments
4D extension has positive impact in OSSE and real observation (low resolution) framework
4D EnVar does have slower convergence
As configured, 4D EnVar was 40% more expensive than 3D hybrid (caveats being different iteration count, low resolution and machine variability)
TLNMC (balance constraint) over all time levels quite expensive
I/O potential issue, optimization is needed prior to implementation.
Option to post-process ensemble prior to use in assimilation in subsequent cycle

9. (4D) Incremental Analysis Update
In addition to TLNMC, we currently utilize full-field digital filter initialization in the GFS
Can have undesirable impacts being a full field filter
IAU (Bloom et al.) has been used at GMAO and elsewhere as alternative to DFI
Increment is passed to model throughout the window as a forcing term
However, this has typically been done using a 3D increment (rescaled) through the window
4D EnVar lends itself to a minor modification of the IAU procedure to use the 4D increment
This also provides a mechanism for passing the 4D solution to the model
Perhaps a way to help spin up/spin down of clouds, precipitation, etc.
Promising preliminary results in EnKF context, work underway in 4D hybrid EnVar context

10. EnKF-IAU noticeably better
EnKF-IAU and EnKF-DFI
EnKF-IAU noticeably better

11. Compare 6-h forecasts to EC analysis
EnKF-IAU forecasts closer to EC analysis, esp for winds
Temperature
Uwind (Vwind is similar to Uwind)

12. Accounting for under-represented source of uncertainty
Current operations
Multiplicative inflation – Helps compensate for finite-sized ensemble
Additive perturbations – Helps compensate for lack of consideration of model uncertainty
Proposed configuration for T1534 SL (3072 x 1536) GFS
Keep multiplicative inflation
Reduce additive perturbations from 32 to 5%
Add stochastic physics in model
SPPT -- Stochastic Physics Perturbation Tendency
SKEB – Stochastic Energy Backscatter
VC – Vorticity Confinement
SHUM – Boundary Layer Humidity perturbations

13. Better spread behavior (2014042400)
Current
Stoch. Phy
Current operations
Spread too large
Spread decays and recovers
Stochastic Physics
Spread decreased overall (consistent with error estimates)
Spread grows through assimilation window
Wind Tv
3HR
6HR
9HR Q

14. PROPOSED - Dual-Res Coupled Hybrid Var/EnKF Cycling
Generate new ensemble perturbations given the latest set of observations and first-guess ensemble
member 1
forecast
member 1
analysis
T574L64
EnKF
member update
member 2
forecast
member 2
analysis
recenter analysis ensemble
Ensemble contribution to background error covariance
member 3
analysis
member 3
forecast
GSI 4D
Hybrid Ens/Var
Replace the EnKF ensemble mean analysis
and inflate
high res
forecast
T1534L64
Low-to-high
res analysis
Previous Cycle
Current Update Cycle

15. Tests for FY2015 Implementation
Analysis on ensemble grid at T574 instead of hi-res deterministic grid T1534
Configure update and outer loop
Quasi-outer loop (rerun of nonlinear model as in 4D Var)
Re-scale ensemble perturbations for successive outer-loops?
Continue to investigate use of IAU to force 4D increment into model
Test role of stochastic physics as replacement for additive inflation
Relaxed binning criteria within the assimilation window
Lengthen assimilation window (backward) in catch-up cycle
Improved (or at least tuned) localization
Optimize static B for 4D hybrid
Computationally (current static B is bottleneck in code)
Add temporal information (FOTO)
Weights, including scale-dependence
Increase role of ensemble (to 100%?)
4D EnVar tentatively scheduled to be part of GDAS/GFS upgrade in late 2015