1. 1 Thomas Karl Director National Oceanic and Atmospheric Administration National Climatic Data Center AMS 88 th Annual Meeting Town Hall Meeting January 21,2008 NOAA Reanalysis User Needs AMS Town Hall Meeting

2. 2 Outline Purpose of Meeting Overview of NOAA Reanalysis Efforts Climate Forecast System Reanalysis and Reforecast (CFSRR): –Data Availability Plans –Data Access Optimization User Input and Open Discussion

3. 3 Has the proposed file organization and groupings reflect current user needs and expectations? Which data need to be placed on-line given e.g., 150TB of disk? Are we on the right track? Do we need to re-group data sets to create needed products – such as initialization files, analysis, or by forecast projections? Can some other mechanism (i.e. LDM) provide some CFSRR output to users in near real-time, shifting the load from the archive? NCDC and NCEP are committed to provide the highest resolution if possible. Tonight’s discussion is a focus on user needs tempered by available resources. Tonight’s discussion is a focus on user needs Open Forum and User Input

4. 4 New NOAA Reanalysis Projects 1) Historical SFC Reanalysis (Compo et al.,) –1850 to present. ~60TB 2) Post WW-II Reanalysis (NCEP) –1944 to present. ~235TB 3) Climate Forecast System Reanalysis and Reforecast (CFSRR) Project (Saha et al.) – 1978-2008 Reanalysis and Seasonal Reforecast: 915TB (Grib1) Development of a Reanalysis Clearinghouse –Capability for an “on-going analysis of the climate system”

5. 5 CFSRR Overview (Saha, EMC/NCEP) Coupled Climate Forecast System (CFS) Reanalysis and Reforecast (CFSRR) (Atm - Ocn - Land – SeaIce) has two parts: –Reanalysis: 31-year (1979-2009) T384L64 (~32 km and 64 hybrid layers for the atmosphere, 0.5 0 and 40 levels for the ocean, 4 soil levels) –Reforecast: 28-year (1982-2009) T126L64 (~32 km and 64 hybrid layers for the atmosphere, 0.5 0 and 40 levels for the ocean, 4 soil levels) –6 hourly Reforecast for 1 year NCDC and NCEP are seeking community input for archive and access priorities to this massive dataset Approximately 915 Terabytes in GRIB1. Note: this presentation assumes conversion is needed to Grib1 as many user applications cannot yet deal with GRIB2 directly (including NCDC).

6. 6 Proposed CFS Reanalysis Data CFS High Resolution Initial Conditions: 72 TB CFS Low Resolution Initial Conditions: 35 TB Full Data Ingest : 46TB (satellite radiances, RAOB, in-situ, etc.) Pgbh Pressure Grib Files 0.5 x 0.5 Hourly: 91TB Flxf T382 Gaussian Hourly: 25TB Ocnh 0.5 x 0.5 Hourly: 18TB Diabf 1.0 x 1.0 Hourly: 27TB Ipvh 0.5 x 0.5 Hourly: 17TB Monthly: 1TB TOTAL: 332 TB

7. 7 1.Siganl : 3-D Hybrid Analysis 2.Sfcanl : Surface Analysis 3.Ocnanl : 3D Ocean Analysis 4.Pgbh : 3-D Pressure level data 5.Flxf : SFC fluxes, radiative fluxes, precip. 6.Ocnh : 3-D Ocean data 7.Ipvh : 3-D Isentropic level data 8.Diabf : Diabatic Heating, Moistening rates, etc. 9.Egyh : Energetics, u ’ v ’, TKE, etc. Proposed CFS Reanalysis Hi- Resolution File Level Organization T382L64 + Ocean (1/4 x ½) 6 hourly = 72TB T126L64 + Ocean (1/2 x 1) 6 hourly = 35 TB 0.5 x 0.5 degree Hourly= 91TB T382 Gaussian Hrly= 25TB 0.5 x 0.5 degree Hourly= 18TB 1.0 x 1.0 6-Hourly: 27TB 0.5 x 0.5 degree Hourly: 17TB 0.5 x 0.5 Monthly: 1 TB

8. 8 CFS 28-Year Reforecast Data 1982-2009 6-hourly Pressure Grib: 412TB 6-hourly Ocean : 102TB 6-hourly Ipv: 78TB Monthly and Time-series: 3TB 1.0 x 1.0 degree for first 6 months of forecast 2.5 x 2.5 for next 6 months of forecast TOTAL: 595 TB

9. 9 National Academies Recommendation National Research Council, Board of Atmospheric Sciences and Climate (BASC): “Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts” Recommendation 3.4: The NOAA National Operational Model Archive and Distribution System (NOMADS) should be maintained and extended to include (a) long-term archives of global and regional ensemble forecasting systems and their native resolution, and (b) re-forecast datasets to facilitate post-processing”

10. 10 Open Discussion Proposed Data and System Access Priorities

11. 11 1)Basic ftp and http scripting services to the highest resolution data though the NCDC tape archive services. 2) NOMADS on-line sub-setting services for most requested data (tonight’s discussion). 3) Advanced GIS and other Web Based Portal Services as resources and time permit Issue: Data access may be throttled based on number of concurrent users, I/O restraints, and communications bandwidth at NCDC. CFSRR System Access Prioritization 1

12. 12 Reanalysis Data Access Prioritization 1 Highest Resolution CFSRR data provided –Low Res users can subset or use Post WW-II –Access assisted by user feedback as to file level organization (this meeting!) Historical SFC Reanalysis –60TB (Grib and HDF) FY10 Post WW-II U/A Reanalysis –Low resolution 6-hourly (235TB Grib-1) – When funded

13. 13 T382 Siganl227.1 MB T382 Sfcanl116.8 MB MOM4 1/4 x 1/2 Ocnanl1242.3 MB Pgb (0.5 x 0.5)250 MB Flx T382 Gaussian77 MB Diab (0.5 x 0.5)85 MB Ocn (0.5 x 0.5)56 MB Ipv (0.5 x 0.5)46 MB Egy (0.5 x 0.5)123 MB Monthly Means1.5MB Data Requirements Prioritization (1) CFS Reanalysis Data Highest resolution data to be made available

14. 14 MOM4 1/3 x 1 Ocnanl350.1 MB T126 Siganl25.2 MB T126 Sfcanl12.9 MB Pgb (1.0 x 1.0)60 MB Pgb (2.5 x 2.5)10 MB Ipv (1.0 x 1.0)11.5 MB Ipv (2.5 x 2.5)1.8 MB Flx T126 Gaussian9 MB Flx T62 Gaussian2.2 MB Ocn (1.0 x 1.0)15 MB Egy (1.0 x 1.0)31 MB Egy (2.5 x 2.5)5 MB CFS Reanalysis Data Data Requirements Prioritization (2) Lower Resolution Data

15. 15 FEEDBACK and QUESTIONS Suranjana.Saha@noaa.gov CFSRR PI Glenn.Rutledge@noaa.gov nomads.ncdc.noaa.gov David.Easterling@noaa.gov OPEN DISCUSSION

16. 16 Backup Slides

17. 17 To overcome a deficiency in model data access, some of the Nations top scientists are actively engaged in a grass-roots framework to share data and research findings over the Internet. NCDC, NCEP and GFDL initiated the NOAA Operational Model Archive and Distribution System. NOMADS is a distributed data services pilot for format independent access to climate and weather models and data. Overview

18. 18 foster research within the geo-science communities (ocean, weather, and climate) to study multiple earth systems using collections of distributed data; promote model evaluation and community feedback; develop institutional partnerships and access via distributed open standard technologies. Establish a unified climate and weather model archive providing format independent access to retrospective models; NOMADS Goals The NOAA Operational Model Archive and Distribution System

19. 19 NCEP Dual Ingest & QC Portals GDS and TDS Live Access Server GrADS, Ferret, MatLab, IDL, IDV, Web browsers or any OPeNDAP enabled client Project ACCESS: NASA, GMU, GMU & OPeNDAP Project GALEON Unidata NOMADS Web-Plotter, http, ftp & binary subsetting GIS access (WCS, W*S) NCDC Archive CEOS-Grid NOAA-wide LAS “sister-servers” Exploratory Grid Projects w/ Globus Collaboration Focus The NOMADS System Design Multiple paths to format independent data access

20. 20 NCDC HDSS Access System (HAS = NCDC Archive) NOMADS Model data CLASS Satellite NEXRAD Radar CDO, GIS Services In-situ Data request from access service—eg, NOMADS Data delivery to access service—eg, NEXRAD A very simplified diagram of data flow from NCDC‘s HDSS tape robotics system & incoming sources Other incoming data Major Access Services NOAA Customers