1 Thomas Karl Director National Oceanic and Atmospheric Administration National Climatic Data Center AMS 88 th Annual Meeting Town Hall Meeting January 21,2008 NOAA Reanalysis User Needs AMS Town Hall Meeting

2 Outline Purpose of Meeting Overview of NOAA Reanalysis Efforts Climate Forecast System Reanalysis and Reforecast (CFSRR): –Data Availability Plans –Data Access Optimization User Input and Open Discussion

3 Has the proposed file organization and groupings reflect current user needs and expectations? Which data need to be placed on-line given e.g., 150TB of disk? Are we on the right track? Do we need to re-group data sets to create needed products – such as initialization files, analysis, or by forecast projections? Can some other mechanism (i.e. LDM) provide some CFSRR output to users in near real-time, shifting the load from the archive? NCDC and NCEP are committed to provide the highest resolution if possible. Tonight’s discussion is a focus on user needs tempered by available resources. Tonight’s discussion is a focus on user needs Open Forum and User Input

4 New NOAA Reanalysis Projects 1) Historical SFC Reanalysis (Compo et al.,) –1850 to present. ~60TB 2) Post WW-II Reanalysis (NCEP) –1944 to present. ~235TB 3) Climate Forecast System Reanalysis and Reforecast (CFSRR) Project (Saha et al.) – Reanalysis and Seasonal Reforecast: 915TB (Grib1) Development of a Reanalysis Clearinghouse –Capability for an “on-going analysis of the climate system”

5 CFSRR Overview (Saha, EMC/NCEP) Coupled Climate Forecast System (CFS) Reanalysis and Reforecast (CFSRR) (Atm - Ocn - Land – SeaIce) has two parts: –Reanalysis: 31-year ( ) T384L64 (~32 km and 64 hybrid layers for the atmosphere, and 40 levels for the ocean, 4 soil levels) –Reforecast: 28-year ( ) T126L64 (~32 km and 64 hybrid layers for the atmosphere, and 40 levels for the ocean, 4 soil levels) –6 hourly Reforecast for 1 year NCDC and NCEP are seeking community input for archive and access priorities to this massive dataset Approximately 915 Terabytes in GRIB1. Note: this presentation assumes conversion is needed to Grib1 as many user applications cannot yet deal with GRIB2 directly (including NCDC).

6 Proposed CFS Reanalysis Data CFS High Resolution Initial Conditions: 72 TB CFS Low Resolution Initial Conditions: 35 TB Full Data Ingest : 46TB (satellite radiances, RAOB, in-situ, etc.) Pgbh Pressure Grib Files 0.5 x 0.5 Hourly: 91TB Flxf T382 Gaussian Hourly: 25TB Ocnh 0.5 x 0.5 Hourly: 18TB Diabf 1.0 x 1.0 Hourly: 27TB Ipvh 0.5 x 0.5 Hourly: 17TB Monthly: 1TB TOTAL: 332 TB

7 1.Siganl : 3-D Hybrid Analysis 2.Sfcanl : Surface Analysis 3.Ocnanl : 3D Ocean Analysis 4.Pgbh : 3-D Pressure level data 5.Flxf : SFC fluxes, radiative fluxes, precip. 6.Ocnh : 3-D Ocean data 7.Ipvh : 3-D Isentropic level data 8.Diabf : Diabatic Heating, Moistening rates, etc. 9.Egyh : Energetics, u ’ v ’, TKE, etc. Proposed CFS Reanalysis Hi- Resolution File Level Organization T382L64 + Ocean (1/4 x ½) 6 hourly = 72TB T126L64 + Ocean (1/2 x 1) 6 hourly = 35 TB 0.5 x 0.5 degree Hourly= 91TB T382 Gaussian Hrly= 25TB 0.5 x 0.5 degree Hourly= 18TB 1.0 x Hourly: 27TB 0.5 x 0.5 degree Hourly: 17TB 0.5 x 0.5 Monthly: 1 TB

8 CFS 28-Year Reforecast Data hourly Pressure Grib: 412TB 6-hourly Ocean : 102TB 6-hourly Ipv: 78TB Monthly and Time-series: 3TB 1.0 x 1.0 degree for first 6 months of forecast 2.5 x 2.5 for next 6 months of forecast TOTAL: 595 TB

9 National Academies Recommendation National Research Council, Board of Atmospheric Sciences and Climate (BASC): “Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts” Recommendation 3.4: The NOAA National Operational Model Archive and Distribution System (NOMADS) should be maintained and extended to include (a) long-term archives of global and regional ensemble forecasting systems and their native resolution, and (b) re-forecast datasets to facilitate post-processing”

10 Open Discussion Proposed Data and System Access Priorities

11 1)Basic ftp and http scripting services to the highest resolution data though the NCDC tape archive services. 2) NOMADS on-line sub-setting services for most requested data (tonight’s discussion). 3) Advanced GIS and other Web Based Portal Services as resources and time permit Issue: Data access may be throttled based on number of concurrent users, I/O restraints, and communications bandwidth at NCDC. CFSRR System Access Prioritization 1

12 Reanalysis Data Access Prioritization 1 Highest Resolution CFSRR data provided –Low Res users can subset or use Post WW-II –Access assisted by user feedback as to file level organization (this meeting!) Historical SFC Reanalysis –60TB (Grib and HDF) FY10 Post WW-II U/A Reanalysis –Low resolution 6-hourly (235TB Grib-1) – When funded

13 T382 Siganl227.1 MB T382 Sfcanl116.8 MB MOM4 1/4 x 1/2 Ocnanl MB Pgb (0.5 x 0.5)250 MB Flx T382 Gaussian77 MB Diab (0.5 x 0.5)85 MB Ocn (0.5 x 0.5)56 MB Ipv (0.5 x 0.5)46 MB Egy (0.5 x 0.5)123 MB Monthly Means1.5MB Data Requirements Prioritization (1) CFS Reanalysis Data Highest resolution data to be made available

14 MOM4 1/3 x 1 Ocnanl350.1 MB T126 Siganl25.2 MB T126 Sfcanl12.9 MB Pgb (1.0 x 1.0)60 MB Pgb (2.5 x 2.5)10 MB Ipv (1.0 x 1.0)11.5 MB Ipv (2.5 x 2.5)1.8 MB Flx T126 Gaussian9 MB Flx T62 Gaussian2.2 MB Ocn (1.0 x 1.0)15 MB Egy (1.0 x 1.0)31 MB Egy (2.5 x 2.5)5 MB CFS Reanalysis Data Data Requirements Prioritization (2) Lower Resolution Data

15 FEEDBACK and QUESTIONS CFSRR PI nomads.ncdc.noaa.gov OPEN DISCUSSION

16 Backup Slides

17 To overcome a deficiency in model data access, some of the Nations top scientists are actively engaged in a grass-roots framework to share data and research findings over the Internet. NCDC, NCEP and GFDL initiated the NOAA Operational Model Archive and Distribution System. NOMADS is a distributed data services pilot for format independent access to climate and weather models and data. Overview

18 foster research within the geo-science communities (ocean, weather, and climate) to study multiple earth systems using collections of distributed data; promote model evaluation and community feedback; develop institutional partnerships and access via distributed open standard technologies. Establish a unified climate and weather model archive providing format independent access to retrospective models; NOMADS Goals The NOAA Operational Model Archive and Distribution System

19 NCEP Dual Ingest & QC Portals GDS and TDS Live Access Server GrADS, Ferret, MatLab, IDL, IDV, Web browsers or any OPeNDAP enabled client Project ACCESS: NASA, GMU, GMU & OPeNDAP Project GALEON Unidata NOMADS Web-Plotter, http, ftp & binary subsetting GIS access (WCS, W*S) NCDC Archive CEOS-Grid NOAA-wide LAS “sister-servers” Exploratory Grid Projects w/ Globus Collaboration Focus The NOMADS System Design Multiple paths to format independent data access

20 NCDC HDSS Access System (HAS = NCDC Archive) NOMADS Model data CLASS Satellite NEXRAD Radar CDO, GIS Services In-situ Data request from access service—eg, NOMADS Data delivery to access service—eg, NEXRAD A very simplified diagram of data flow from NCDC‘s HDSS tape robotics system & incoming sources Other incoming data Major Access Services NOAA Customers