1. Obs-nudging-FDDA
An overview

2. Obs-nudging-FDDA Overview
Obs-nudging-FDDA vs. NWP forecast systems
Overview
Restarts and cold starts
Cycling
RT vs. Climo and Ensembles
Real Time Four Dimensional Data Assimilation
Has evolved to include other variations
CFDDA
Climate FDDA
E-RTFDDA
Ensemble RTFDDA
Obs-nudging-FDDA inputs and outputs
Background and history within NSAP

3. RTFDDA: What is it?
Real Time Four Dimensional Data Assimilation system
3-D plus time system combining observational analysis including obs nudging and a forecast model
Can use either MM5 or WRF for the forecast “engine”

4. RTFDDA “the science” concept
PSU/NCAR MM5 or NCEP/NCAR WRF - based Four Dimensional Data Assimilation (FDDA) system run in a rapid update cycle.
Concept of dynamically combining observational and model data in space and time, first developed by David Stauffer and Nelson Seaman (Stauffer & Seaman 1994).
Uses newtonian relaxation technique to “nudge” the model solution towards the observed values.
Provides continuous analyses and short term forecasts (~48hrs) every 3hrs.

5. New 12 - 48 h forecasts every 1 - 12 hrs, using all obs up to “now”
"Obs-nudging"
FDDA
All WMO/GTS
GOES
Radars
New h forecasts every hrs, using all obs up to “now”
obs t
Forecast
FDDA
WRF/MM5- RTFDDA
Multi-scale Modeling
Wind Profs
Cold start
MESONETs
Etc.
ACARS
TAMDAR

6. Example Model Domains Cold Regions Test Center (CRTC)
Nested model domains of higher resolution
Most 4DWX applications use 3 or 4 nested domains
Domain1 is 30km, nest resolution is 3:1 ratio in MM5
Domain 2 is 10km
Domain 3 is 3.3km
Domain 4 is 1.1km

7. RTFDDA: Why is it different?
Cold Start vs. Continuous
Most forecast models run 2-4 times per day, with each run “starting over”. This is a cold-start, and the model equations need time to spin-up and stabilize some parameters, esp. moisture.
RTFDDA runs “continuously”, a cold-start is only done once a week, so limited model spin-up time is involved. How often to cold-start is a system-tuning parameter, but weekly is typical for real-time systems.

8. Cycles and Restarts
Obs-nudging-FDDA systems do not really run continuously, they stop and restart
Each stop/restart is called a “cycle”
Cycles are defined in the project setup/configuration and can be anything from 1 hour to 7 days or more. Typical real-time cycles are 3-hours
Each cycle, the model reads a “restart” file from the previous cycle to re-initialize the model state variables, so that the model does not have to start it’s equations & variables from initialization, as it does at a cold-start time.
By doing a restart, the cycle allows time for a forecast to be generated, and for new observations to be collected and processed for the next cycle.

9. Obs-nudging-FDDA cycles
Current ATEC RTFDDA configurations run cycles every 3 hours, at 02Z, 05Z, 08Z, etc. This timing is based on data availability.
1, 2 or 6 hour cycle intervals are sometimes used
For rapid real-time updates (1 or 2 hour cycles)
For longer forecast output options (6 hour cycles)
Precise cycle timing depends on
cluster compute-power
forecast length goals
data availability
Output products generally use “the latest available” for the requested valid time
This assumes that the latest available is the most accurate
Each cycle’s output may be saved/viewed if desired

10. Anatomy of a Cycle RTFDDA
05:00
06:00
07:00
08:00
Pre-process
Obs
FDDA Final Analysis
MM5 running
Process Add’l
Prelim + Forecast …
MM5 running…
Valid Time: The time at which this model output is computed to be valid.
FDDA Final Analysis: The last time this valid-time is processed. All obs are included in the obs nudging here as the model is running. Any that come later are *not* included
Preliminary Analysis: The “current” hours, some obs are available; this valid time will be re-processed next cycle with more obs.
Forecast: No observations are available (valid times in the future); these valid times will be re-processed several more times as obs become available.
For example, a 05Z cycle:
05:30 – Starts preprocessing, including collecting NAM/obs data
05:50 – Starts running MM5 for FDDA final analysis hours 02, 03, 04Z
06:15 – Finishes final analysis; starts re-processing obs data
06:30 – Starts running MM5 for prelim+forecast; Any obs which are valid during these hours will be included (usually 05Z, a few 06Z), then straight forecast for the remaining hours hours of output for this phase
08:30 – 08Z cycle start
Cycle Starts
Longer or shorter for different cycle intervals

11. Cycle Output RTFDDA Valid Time Cycle 05Z 08Z 11Z 14Z 17Z 20Z 02 03 0406 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22
05Z
08Z
11Z
Cycle
14Z
17Z
05Z cycle – FDDA final analysis output at 06:15
07Z output at about 07Z (caught up to real-time)
08Z to next day – finished by 08:30 for 08Z cycle
Similarly for each 3-hour cycle
For example, for a valid time of 18Z:
- 13hr forecast from 05Z
- 10 hr forecast from 08Z
- 7 hr forecast from 11Z
- 4 hr forecast from 17Z
(after-the-fact) a final analysis from 20Z cycle
20Z
Final Analysis
Preliminary Analysis
Forecast

12. Cycle Time

13. Obs-nudging-FDDA: Why is it different?
Observation Analysis
Most models will incorporate observations using analysis nudging at their cold-start time only (0Z, 6Z, 12Z, etc), sometimes using a spin-up data assimilation leading up to that time
Obs-nudging-FDDA incorporates observations as it is running in real-time, at any time they are available
This is a very simplified look at the differences! Remember “data assimilation is obs nudging” and “cycling”…

14. Obs-Nudging-FDDA: Various Flavors
RT-FDDA: RealTime FDDA
Runs with real-time cycles
Rapid (1-6hr cycles) analysis updates
Short term forecasts (24-48hrs)
High resolution nested domains
C-FDDA: Climatology FDDA
Runs for historical time period (days or months)
Analysis only, no forecasts (generally)
Years of climatology are combined for statistical output (mean July temperatures, etc)
E-RTFDDA: Ensemble RT-FDDA
Ensemble of model configurations (WRF, MM5, IC/BC variations, etc)
Real-time cycles
Analysis plus forecasts
High resolution nested domains (but usually not as much detail as single RTFDDA system)
Member outputs are combined for statistical output (mean, variance of temperature at Station #1, etc)
E-CFDDA: Ensemble C-FDDA
Coming Soon! No-one has computers or disks big enough (yet!!!)

15. Obs-nudging-FDDA: Input Data
All Mesoscale NWP systems need input data, they do not start from zero.
All the variations of NSAP’s NWP systems need the same classes of input data
Depending on what type of job you are running, and where the domains are located and the goals of your system design, etc, the input data sources will be different.
But the similarities are greater than the differences!

16. Input Data: IC/BC
Obs-nudging-FDDA requires a low-resolution model to initialize.
These data are used for Initial and Boundary Conditions for the mesoscale model (MM5 or WRF).
RTFDDA uses NAM or GFS, generally, but other data sources have occasionally been used.
CFDDA uses NNRP due to the long time scales needed.
A case study (run one particular day of one particular year for special scientific study) will generally use NAM or GFS if it is available in the archive.
NAM: North American Meso Model (from NCEP)
GFS: Global Forecast System Model (from NCEP)
NNRP: NCAR/NCEP ReAnalysis Product
NCEP: National Centers for Environmental Prediction

17. Input Data: Obs CFDDA uses ADP obs from the NCAR archives
Observations are critical to the performance of any of the Obs-nudging-FDDA systems (RTFDDA, CFDDA or E-RTFDDA).
Observation datasets for RTFDDA:
WMO (metars, soundings) – (LDM from NOAAPort, as backup to MADIS)
Range obs (scp from ATEC range machines)
Satellite derived winds (included in MADIS)
ACARS (included in MADIS)
MesoWest (included in MADIS)
The MADIS dataset from FSL combines most of these except the ATEC range data (or other special data, such as the Torino or Athens obs)
CFDDA uses ADP obs from the NCAR archives
None of these will cause a “failure”/abort of RTFDDA, if they are missing, but will impact it’s accuracy and performance!
ADP: Automated Data Processing (historical obs dataset from NCEP)
MADIS: Meteorological Assimilation Data Ingest System
WMO: World Meteorological Organization
ACARS: Aircraft Communication Addressing and Reporting System
FSL: Forecast Systems Laboratory (part of NOAA)
LDM: Local Data Manager

18. Typical RTFDDA dataflow
web server
Web accessible Output Products
UGUI gifs
NAPS
MEDOC
FDDA sites
data host
MDV server/files
JVIZ. etc
Gridded Climatology Files
data input host
ETA
GFS
WMO
Satwinds
NPN
BLP
ACARs
RAWs
MADIS
<range>
Other
NCEP
Ranges
NOAA
NOAAPort
FSL
model cluster
RTFDDA cycles
Output Products

19. ATEC RTFDDA operations
At NCAR:
MM5-RTFDDA --- ATC, CRTC, DPG, EPG, RTTC, WSMR, YPG
Safari requests – PMRF, etc
gs-c2
das-input*
gs-c3
4dwx.org
smac-c2
Web Pages
dpg-c2
crtc-c3
range DAS
range DAS
wsmr-c2
ypg-c2
atc-c2
Web Pages
JVIZ
At Ranges:
WRF-RTFDDA --- ATC, CRTC, DPG, WSMR, YPG

20. RTFDDA Operations
ATEC is the largest user of RTFDDA operationally, but not the only user
Specific operational setup may vary, depending on cluster site, user output needs, data input requirements, etc.
The RTFDDA system on any cluster for any job will be very similar in setup and configuration

21. Other Operations RTFDDA CFDDA Pentagon Shield
Several DTRA (SLC, Athens and Torino Olympics) and NGIC special operations
FAA winter program
UAE rainfall project
Wyoming rainfall project
Saudi rainfall project
West Africa demonstration project
IAF/TAU demonstration program
NASA NYC remote sensing study program
CFDDA
DTRA global climatology
NGIC regional climatology tool (GCAT)
ATEC range climatology

22. Background and History: 4DWX and modeling
ATEC project started in 1996/97
High resolution forecasting was a primary component of the program
Initial capabilities included a twice-a-day MM5 forecast, plus a retrospective FDDA run on an SGI for 3 ranges
Goal was two-fold:
A high resolution mesoscale forecast for operations, including running coupled applications that required met/weather inputs
A 3-D range climatology computed in “real-time”, year after year.

23. Background and History: RT-FDDA
RT-FDDA system design was developed to address both needs
3-D data assimilation system provided the “best representation of the state of the atmosphere” over the range
Accurate, high resolution forecasts for range users:
Forecasts
Current conditions on the range (including where there are no instruments)
Input weather data for coupled applications such as transport/dispersion, sound propagation, rocket trajectories, parachute drop, etc
Initial RTFDDA system operating for DPG in late 2000, deployed incrementally to 4 other ranges by 2002
Many special operations have been supported throughout the years
EPG and RTTC added in 2006
WRF-RTFDDA added in 2006

24. Background and History: Influence on System Design Decisions
The design of the RT-FDDA cycles and the C-FDDA data assimilation are driven by both scientific principles and user needs
Data availability (input) and user requirements (output) drive choices in timing and system selections throughout the systems
For example:
3 hour cycles:
the GOES satellite winds were originally available every 3 hours (now hourly)
the original clusters could produce a reasonable forecast length in 3 hours (24-30 hours)
mesoscale phenomena timescales also supported a 3-hour cycle
Data sparse regions (UAE), or users needing longer forecasts (PMRF) would get a 6-hour cycle setup
Data rich areas, or users interested in rapid updates willing to trade a shortened forecast length (or as computers gained computational power), would run a 1-hour cycle (a DPG-PWIDS demonstration)

25. Background and History: Climatology
Climatology was an original goal of the ATEC project, and was started with an FDDA system running retrospectively each day in 1997 (for DPG)
NGIC was the first sponsor to develop a more generalized tool to use the cluster computer power and historical climate data to run high-resolution climatologies (NGIC-GCAT)
ATEC has an historical record of more than 10 years of FDDA and RTFDDA/final analysis data from Dugway
is it comparable to a GCAT/CFDDA job run for 10 years?
Why or why not?
Was it a reasonable goal when it was started?
Is there still benefit to archiving the real-time analysis output, when a tool like CFDDA is available?

26. Example System www.4dwx.org/images/atc/rtfdda_gmod/model.html
There is a convention for naming and finding RTFDDA output and webpages!
Where might the WSMR MM5 job output be found?
A “job” generally refers to a configuration, which includes the geographic location, the cluster it’s running on, all the options selected, real-time or climo or case-study, ensemble or not,etc.
Each job will generally produce some web graphics, and some status reports, and other optional output, depending on the configuration.

27. Sample Web Page output, showing various product choices

28. “Range”
Cycle Time
Output Valid Time
Fcst Hours

29. Questions What cluster is this system running on?
How many domains are configured?
What is the typical forecast length produced?
How often is it cycling?
Is it running MM5 or WRF?