1. WoF Modeling Meeting 2/4/13 1 William. M. Lapenta Acting Director Environmental Modeling Center NOAA/NWS/NCEP Geoff DiMego, John Derber, Yuejian Zhu, Hendrik Tolman, Vijay Tallapragada, Shrinivas Moorthi, Mike Ek, Mark Iredell, Suru Saha The NOAA Operational Numerical Guidance System: Meeting the WoF and WRN Challenge

2. AMS Future of the Weather Enterprise 11/27/12 NOAA Center for Weather and Climate Prediction: “A Game Changer ” 2 A.K.A.—the new building…. Four-story, 268,762 square foot building in Riverdale, MD will house 800+ Federal employees, and contractors 5 NCEP Centers (NCO, EMC, HPC, OPC, CPC) NESDIS Center for Satellite Applications and Research (STAR) NESDIS Satellite Analysis Branch (SAB) OAR Air Resources Laboratory Includes 465 seat auditorium & conference center, library, deli, fitness center and health unit Includes 40 spaces for visiting scientists Represents a “Game Changer” in our ability to do business

3. WoF Modeling Meeting 2/4/13 NOAA Operational Numerical Guidance Supports the Agency Mission –Numerical Weather Prediction at NOAA  Related to ability to meet service-based metrics (below) –National Weather Service GPRA* Metrics (* Government Performance & Results Act)  Hurricane Track and Intensity  Winter Storm Warning  Precipitation Threat  Flood Warning  Marine Wind Speed and Wave Height –Customer Service Provider  Operational numerical guidance provides foundational tools used by Government, public and private industry to Improve public safety, quality of life and make business decisions that drive US economic growth 3 Lead Time and Accuracy!

4. WoF Modeling Meeting 2/4/13 4 DRAFT Storm Prediction Center Potential Products and Services

5. WoF Modeling Meeting 2/4/13 DRAFT Storm Prediction Center Desired Numerical Guidance Attributes 5 Inside every HRRRE member Movable domain update SSEF: 1 km movable regional domain 20-30 member storm scale ensemble advanced state-of-the-science DA run every 1 hr with forecasts to 18-24 hr focused on “severe weather of the day” areas

6. AFWA NWP 11 Sept 2012 Plans for a National Mesoscale Ensemble System Convergence of NAM, RAP, HRRR and SREF between 2016 and 2018 System must meet requirements of today and future –Provide NextGen Enroute AND terminal guidance –Provide PROBABILITY guidance with full Probability Density Function specified –Address Warn-on-Forecast as resolutions evolve towards ~1 km Core elements –Common data assimilation system –Code Infrastructure and management system Potential Attributes of the North American Rapid Refresh Ensemble (NARRE) –Hourly updated with forecasts to 24 hours –Control data assimilation cycles with 3 hour pre- forecast period –84 hr forecasts are extensions of the 00z, 06z, 12z, & 18z runs 6

7. WoF Modeling Meeting 2/4/13 7 Air Quality WRF NMM/ARW Workstation WRF WRF: ARW, NMM NMMB GFS, Canadian Global Model Regional NAM WRF NMMB North American Ensemble Forecast System Hurricane GFDL HWRF Global Forecast System Dispersion ARL/HYSPLIT For eca st Severe Weather Rapid Refresh for Aviation Climate Forecast System Short-Range Ensemble Forecast NOAA’s Operational Numerical Guidance Suite GFS MOM4 NOAH Sea Ice NOAH Land Surface Model Coupled Global Data Assimilation Oceans HYCOM WaveWatch III NAM/CMAQ 7 Regional DA Satellites + Radar 99.9% ~2B Obs/Day NOS – OFS Great Lakes Northern Gulf of Mex Columbia R. Bays Chesapeake Tampa Delaware Space Weather ENLIL Regional DA Sea Nettle Forecast 7

8. WoF Modeling Meeting 2/4/13 Number of Nodes High Water Mark 2010 24-h Snapshot 20 August 2012 Time of Day (UTC) 00 06 1218 00 Numerical Guidance Suite Execution on the Operational NOAA Supercomputer 8 CFS NAM GFS GEFS SREF

9. WoF Modeling Meeting 2/4/13 Operational Compute Resource Consumption (%) by Component Percent of Total CPU Numerical Guidance System Component Statistics accumulated over a 24-h period Total 64.9% of production machine NAM, CFS and SREF compose 29.7% of production 9

10. WoF Modeling Meeting 2/4/13 Global Data Assimilation System Upgrade Hybrid system –Most of the impact comes from this change –Uses ensemble forecasts to help define background error NPP (ATMS) assimilated –Quick use of data 7 months after launch Use of GPSRO Bending Angle rather than refractivity –Allows use of more data (especially higher in atmos.) –Small positive impacts Satellite radiance monitoring code –Allows quicker awareness of problems (run every cycle) –Monitoring software can automatically detect many problems Partnership between research and operations –(NASA/GMAO, NOAA/ESRL, Univ OK, and NOAA/NCEP) Consolidation across systems –Unify operational data assimilation system for global, regional and hurricane applications –Cost effective—O&M –Configuration management 10 Implemented 22 May 2012

11. WoF Modeling Meeting 2/4/13 11 NCEP Closing the International Gap June, July, August 500hPa Geopotential RMSE  NCEP achieved significant improvement in 2012 for day 3 and beyond  NCEP is now similar to UKMO skill in this metric and AC Meteo-Fr CMC NCEP UKMO ECMWF 2011 2012 Solid line lower than dashed indicates improvement between 2011 and 2012 NCEP Only System to show improvement between 2011 and 2012

12. WoF Modeling Meeting 2/4/13 Gridpoint Statistical Interpolation (GSI) Data Assimilation System 12 Unified variational data assimilation (DA) system Global and regional applications Weather and climate Operational system being used by NOAA (GFS, NAM, RTMA, HWRF, RAP…) NASA (GMAO global) AFWA (Testing in progress) Distributed development: NCEP/EMC, NASA/GMAO, NOAA/ESRL, NCAR/MMM, … A community code used in NOAA operations and research Supported by NCEP, NASA, DTC Disciplined code management policy and procedures required to incorporate changes across user organizations http://www.dtcenter.org/com-GSI/users/index.php

13. WoF Modeling Meeting 2/4/13 Goals of Community GSI Efforts Provide current operational GSI capabilities to the research community (O2R) Provide a framework for distributed development of new capabilities & advances in data assimilation Provide a pathway for data assimilation research to operations process (R2O) Provide rational basis to operational centers and research community for enhancement of data assimilation systems 13

14. WoF Modeling Meeting 2/4/13 Adding New Analysis Variables to GSI No longer a difficult procedure o Thanks to GSI Bundle infrastructure (via NCEP/NASA collaboration) Generalized structure for analysis, guess, state, and control variables o e.g., hydrometeors, vertical velocity, double moment microphysics variables, etc. Control through table in the anavinfo file Greatly reduces the number of routines needed to be modified Modify table(s) + possible additional model I/O steps + observation ingest + forward operator + background error Can also disable the conventional large-scale balance constraints present in the NMC derived background errors

15. WoF Modeling Meeting 2/4/13 15 North America Model (NAM) Implemented 18 October 2011 NEMS based NMM Outer grid at 12 km to 84hr Multiple Nests Run to ~60hr –4 km CONUS nest –6 km Alaska nest –3 km HI & PR nests –1.3km DHS/FireWeather/IMET (to 36hr) Rapid Refresh (RAP) Implemented 1 May 2012 WRF-based ARW Use of GSI analysis Expanded 13 km Domain to include Alaska Experimental 3 km HRRR RUC-13 CONUS domain WRF-Rapid Refresh domain – 2010 Original CONUS domain Experimental 3 km HRRR NOAA Operational Mesoscale Modeling for CONUS:

16. WoF Modeling Meeting 2/4/13 Numerical Guidance to Support a NWS Operational Warn on Forecast Capability S. Lord – NWS/OST and NCEP/EMC Staff V4.0

17. WoF Modeling Meeting 2/4/13 Introduction Goal: field an operational, credible and skillful WoF capability by 2025 Combines forecast requirements for –NWS severe weather forecasts and warnings –Aviation (NEXTGEN) –Daily general weather guidance to 3 days, including QPF Closely allied with Hurricane forecast technology (HFIP) Effort exceeds decade-long development and implementation cycle –Current operational system provides look at the future pieces –Incremental system development provides a risk-managed path for operations –With sustained development of the operational pieces, and sufficient HPC, NOAA (OAR and NWS) can achieve the WoF goal Development of operational systems provides enhancement and generalized capabilities to incorporate R&D and improve capabilities (including forecast skill) Improved skill must be documented with appropriate performance measures (TBD) HPC capability growth is essential to meet goal –Adequate resolution for predicting high impact weather events poses largest operational HPC requirements Forecast model Data assimilation Ensemble & postprocessing system –Advanced nature of severe weather problem requires larger ratio of R&D HPC to operational HPC –Constant $$ HPC growth and new technology (e.g. GPU, MIC, etc) will be necessary but insufficient

18. WoF Modeling Meeting 2/4/13 International Thrust – UKMO 2014 ModelResolution (km) Forecast Length Frequency (Members) Other details UK1.53683d var UK ensemble2.2364(12)Global perturbed boundaries and init. Conditions Global17144 72 2222 Hybrid 4-d Var Global Ensemble 33724(12)ETKF with Stochastic Physics UK is a very small country! –Area covers approximately Fire Wx nest (next slide) –Weather events less diverse –Regional high resolution NWP is more affordable computationally US global system (planned for 2014) has comparable resolution –SL T1148 (17 km) –Hybrid DA (27 km ensemble) enhances skill to comparability with UKMO US regional system lags in operational implementation (due to HPC constraints) but not technology level

19. WoF Modeling Meeting 2/4/13 19 Global  Continental  Local Nesting Strategy Discussion items –Nesting necessary to save computing; current NAM domain could be expanded to global 10 km –Advantages/disadvantages of same model for global and regional –Applicability (DA) of a multi-model ensemble-based covariance estimate with multiple bias characteristics –Data assimilation at resolution of 3 km or less poses challenges Advanced assimilation of radar reflectivity Consistent integration of radar and satellite info Proper dynamically balanced analysis increments at multiple high resolution scales (10, 3, 1 km) to minimize spin-up/spin-down and maximize evolution of nascent disturbances

20. WoF Modeling Meeting 2/4/13 Numerical Guidance Attributes (1) Analysis and uncertainty products –Hourly refresh but sub-hourly for rapidly changing observables (reflectivity, hydrometeors, wind shear) –1 km resolution for rapidly changing observables –Hybrid analysis provides analysis uncertainty information and ensemble perturbations –Multi-model ensemble-based background (technology not developed) –Analysis variables Surface (sensible wx) variables Clouds Radar reflectivity, hydrometeors 3-D atmosphere Aerosols Skin temperature Precipitation amount and hydrometeors Land & hydrology Eventual coupled system to add marine & coastal analysis (waves, water level, water quality) –Products support forecaster Situational Awareness (SA) MRMS products are an initial capability Enhancement to Rapidly Updating Analysis (RUA) –Combines obs. from »Radar »Satellite »In situ –Expands to hydrological and coastal marine domain –Provides best and consistent geophysical picture from all available information –Catchup cycle for late data produces “Analysis of Record” for verification

21. WoF Modeling Meeting 2/4/13 Numerical Guidance Attributes (2) Initialized from hourly analysis Evolved through RAP, HRRR & NAM development –North American Rapid Refresh Ensemble (NARRE) –High Resolution Rapid Refresh Ensemble (HRRRE) 1-way nested –Technology available in NAM as prototype –1 km resolution where needed –Ensemble-based probabilistic products support Severe convective weather (incl. tornado environment) Aviation and surface transportation Landfalling hurricanes Flash floods –Eventual support for Storm surge Waves Water quality High resolution analysis and nested forecast system provide consistency between real-time SA and forecast evolution

22. WoF Modeling Meeting 2/4/13 Current Ops (NAM-RR)2025 WoF Ops Parent DomainNorth America (NA)Global Nested domains4 children (CONUS, AK, HA, PR-H) 1 grandchild (Fire Wx) Same children (4) Grandchildren (27) - Fire Wx (2) - WoF (5) - Fixed airport (20) Resolution12-13 km (NA) 4 km CONUS, 6 km AK, 3 km HA, PR-H 1 km Fire Wx (grandchild) 10 km Global 3 km children 1 km grandchildren Analysis update6 hourly (NA) 1 hour CONUS 3-6 hour (global) 1 hour (child, grandchild) EnsembleNA: Short-Range Ensemble Forecast (SREF, 21 members, 16 km) Global: GEFS (30 members, 10 km) Child: 30 members, 3 km G’child: 20 members, 1 km Fcst length & frequencyParent: 4X/day to 84 h (NA) Child: 4X/day 60 h (child) G’child; 4X/day to 36 h Global Parent: 4X/day to 240 h at full resolution Child: 4X/day to 84 h; 20X/day to 36 h G’child: 4X/day to 36 h; 20X/day to 6 h Data Assimilation3-D VarParent, Child: 4-D Hybrid (Var-EnKF) G’child: 3-D Hybrid (or more, if affordable) Assimilated variablesPrimarily atmosphereAtmosphere, cloud, radar reflectivity& hydrometeors, aerosol, skin T, land & hydrology Rapid Update Analysis (RUA)Hourly Surface (Real-Time Mesoscale Analysis, RTMA) 2-D fields at 2.5 km 5 min, 1 km 3-D RUA for clouds, radar reflectivity, 1 km 2-D quantitative precipitation estimate (QPE) & surface products; hourly 3- D atmosphere at 2.5 km; land & hydrology, Required computing capability 1Global: 670 X Child: 1734 X G’child: 2373 X Warn on Forecast (WoF) System Comparison Summary

23. WoF Modeling Meeting 2/4/13 Current Ops (NAM-RR)2025 WoF Ops Required computing capability 1Global: 670 X Child: 1734 X G’child: 2373 X System Evolution HPC Gaps Est. HPC Req. HPC RatioNotes Global4146701.6SL can mitigate gap Child17344.21) system too ambitious OR 2) add’l HPC cycles G’child23735.7

24. WoF Modeling Meeting 2/4/13 24 Runtime & optimal node apportionment for operational NMMB nesting with a Fire Wx nest over CONUS (72 nodes) 3 km Puerto Rico nest 3/72 or 4% 1.33 km CONUS FireWx nest 12/72 or 17% 4 km CONUS nest 40/72 or 56% 24 3 km Hawaii nest 4/72 or 5% 12 km parent 8/72 or 11% 6 km Alaska nest 5.72 or 7% 1-way nests solved simultaneously WRF-NMM takes 3.6 times longer than NEMS-NMMB to run Fire Wx nest Increased nest capability by adding processors!

25. WoF Modeling Meeting 2/4/13 Summary Ensemble-based probabilistic guidance for WoF is a major challenge –Scientific development –Demonstration of system capabilities Current operational capabilities can provide basis for future system –But many changes necessary in response to R&D needs and results HPC is second major challenge –Prototype system exceeds planned availability by factor of 4-6 R&D may suggest compromises and where to spend HPC resources to achieve required skill affordably

26. WoF Modeling Meeting 2/4/13 Supplementaries

27. WoF Modeling Meeting 2/4/13 2015-2016 North American Rapid Refresh ENSEMBLE (NARRE) NMMB (from NCEP) & ARW (from ESRL) dynamic cores Common use of NEMS infrastructure and GSI analysis Common NAM parent domain at 10-12 km Initially ~6 member ensemble made up of equal numbers of NMMB- & ARW-based configurations Hourly updated with forecasts to 24 hours NMMB & ARW control data assimilation cycles with 3-6 hr pre-forecast period (catch-up) with hourly updating NAM & SREF 84 hr forecasts are extensions of the 00z, 06z, 12z, & 18z runs – for continuity sake. –SREF will be at same 10-12 km resolution as NARRE by then –SREF will have 21 members plus 6 from NARRE for total of 27 NARRE requires an increase in current HPCC funding 27 DiMego

28. WoF Modeling Meeting 2/4/13 28 2017-2018 High Resolution Rapid Refresh ENSEMBLE (HRRRE) Each member of NARRE contains 3 km nests –CONUS, Alaska, Hawaii & Puerto Rico/Hispaniola nests –The two control runs initialized with radar data & other hi-res obs This capability puts NWS/NCEP[+OAR/ESRL] in a position to –Provide NextGen Enroute AND Terminal guidance (FWIS-like) –Provide PROBABILITY guidance with full Probability Density Function specified, hence uncertainty information too –Provide a vehicle to improve assimilation capabilities using hybrid (EnKF+4DVar) technique with current & future radar & satellite –Address Warn-on-Forecast as resolutions evolve towards ~1 km NAM nests are extensions of the 00z, 06z, 12z & 18Z runs. HRRRE requires an increase in HPCC funding over and above that required for the NARRE DiMego