Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS

Key Steps of Scientific Investigations (archivist point of view) Formulate the questions Review the state of understanding Search and discover data Access data Analyzes data Share new findings Archive results Document new understandings

Search and Discover Data How?  Web based Information Server Features –5K+ html pages (metadata) –All datasets are described (530, 20 TB) –Location of all data files in MSS –Access options are identified –Higher level information Catalogs Project specific descriptions Always current dataset descriptions

Features Organization Navigation Archive Navigation Pull down menus Search

Dataset Page Title and Brief description Systematic Navigation Metadata highlights Period of Record Usage Variables Related Sites (NOAA) Contact Person Related Datasets

Archive Content Weather Center Operational Analyses –NCEP beginning in 1976 Many products – global, regional, different resolutions, full atmosphere stack, all surface grids –ECMWF beginning in 1980 Ditto Atmospheric Reanalyses –NCEP/NCAR 1948 – 2001 –NCEP Version II, 1979 – 2000 –ECMWF ERA15, 1979 – 1993 –ECMWF ERA40, 1958 – 2002 –NCEP N.America Reanalysis

Outstanding Features Three different coordinate surfaces Very long analysis, 2+ Terabytes size Unrestricted distribution CD-ROMS are very popular

Countries Receiving Reanalysis CDROMs Highlights Over 8900 CDROMs /2001 Recipients; U.S. 46%, Japan 11%, (Canada, UK) 4%, (Germany, India) 3%, (Australia, S.Korea, Spain, Mexico, Norway, Russia, France) 2%

Archive Content Observational dataset –Comprehensive Ocean-Atmosphere Data Set (COADS), beginning in 1794 –Global land surface observations, beginning in 1930’s, some earlier –Global upper air measurements, beginning in 1946 Many more supporting datasets and analyses

JOSS & SCD Archives, Mutual Support for Science EXAMPLE GCIP: GEWEX Continental-Scale International Project JOSS in situ data model derived soundings.gif images of 2D model data SCD model output data (Eta, MAPS, GEM) model derived soundings The JOSS and SCD metadata for GCIP are well linked. Illustrates dedicated staff efforts. Future: Can/should we develop better ways to insure scientist make the connection?

Access Data MSS – all the data –No restrictions - open to all with NCAR computing accounts –Great service to the SCD computers –Service to UCAR via LANs/WANs

Data Access Directly from Web Information Server –Open to the public –All data are NOT available here Popular collections Smaller datasets 100 Gigabytes total –Now monitoring data downloads

Data Access Customized data packages –Persons without MSS access –Persons needing large datasets –Persons needing subsets of large datasets –One-off jobs, not done in real time Package delivery –FTP download –Media (tape(s), CD-ROM)

TBytes of Data Provided 5.6 TB in 1995 to about 20.5 TB in 2001

Analyze Data Provide software read the data –Simple programs – users must be programmers SCD/VETS provides much more –NCL (graphics, file manipulations, and computations) Community Data Portal –Real time access to subsets of large datasets (SCD and other UCAR archives) –Real time comparison of datasets –Real time basic analysis computations Scientists “still” design much of their own

Share new findings / Archive results Input Elements –Existing archived datasets –Other archives or new field data Scientific Research –High level data processing –Dataset integration and comparison Output Elements –New data products Shared with colleagues Openly distributed to the public

NCEP Operational Analyses blended with QSCAT Satellite data Wind Stress Curl, 01/24/ UTC a)NCEP Operational ONLY b)NCEP + QSCAT swaths c)OI blend of NCEP + QSCAT Blending by Colorado Research Associates We archive all three products. ab c

Key Steps of Scientific Investigations (archivist point of view) Formulate the questions Review the state of understanding Search and discover data Access data Analyzes data Share new findings Archive results Document new understandings