Ground Systems Data in the World Data Center Archives Justin Mabie (NGDC) and Anatoly Soloviev (GCRAS) 2 July

Ground System Data Holdings Solar IndicesMagnetometer 2 Ionosonde

3 Old School Data in Climate Controlled Storage Boxes and folders and files – oh my! Some are neat, some are a mess, some are so old our computers can’t read them.

In the NGDC Warehouse: 4 To get space in the warehouse, we spend countless hours bringing the data together on a single medium and creating inventories and metadata records. In the past, these data have been migrated to film or packed away in boxes. Modern digital data is packed away neatly in our ADIC tape library system, which is cheaper, less labor intensive, and does not require a large climate controlled warehouse.

Geomagnetic Data Ingest System 5 NGDC workflow for geomagnetic observatory data

Solar Data Ingest System NGDC workflow for solar observations

ISO Metadata Workflow Metadata is a real challenge. This workflow shows how NGDC ingests metadata, performs quality control, ports it to the ISO metadata standard, loads it into databases, and disseminates it to our customers.

Northwestern Research Associates USAF Weather Agency Research Lab NWS SWPC Australia Ionospheric Prediction Services Univ. MA, Lowell Center for Atmos. Research Space Environment Technologies Users Include Ionosonde – User Services Real Time Applications

9 NGDC is developing databases so our users can make use of our metadata in ways we can’t think of. We traditionally focus on the needs of the environmental scientist – what about the instrument and hardware developers, the social scientist, the politicians? For further information, contact: GRAILS/ORACLE Databases for Metadata

XML – Metadata Format 10

CDMP – Climate Data Modernization Project CDMP Data Recovery for Magnetometer, Solar, Ionosonde, and Airborne Imagery Example – Geomag: Title: CDMP - Geomagnetism Volume: Approximately 4TB per year of data Storage: ADIC tape library Dissemination: Reduced resolution images will be put on FTP site; full resolution data available by request. STATUS: All efforts De-Funded due to fiscal crisis. Procedure is in place, providing an opportunity for future funding. Notes:  Data volume is very large (many terabytes)  Full filenames are included to reduce NGDC overhead (lesson learned from earlier projects)  Holdings range in date from 1957 – 1983 for Geomag.  NGDC film magnetograms were on track for completion in 10 years. Solar data is near completion, and Ionosonde data is aprroximately 10% modernized. 11

Geomagnetic and Solar Indices Bulletins Title: Geomagnetic Indices Bulletin Description: Monthly publication containing Kp, Ap, Cp, An, As, Am. Solar Indices bulletin contains Sunspot Number, Sunspot Prediction and Solar Radio Flux data. Media: online only NGDC Holdings: 1996 – Present URL: ngdc.noaa.gov/stp/geomag/geoib.html ftp.ngdc.noaa.gov/STP/SOLAR_DATA/Publications/ DATA SETS Printed for r esearch purposes 12

Geomag Observatory Metadata Form Page 1/5 Page 2/5 XML Page Return 13

RSS feeds to notify users of product changes 14 What is RSS? RSS is a way for a user to automatically retrieve information from selected websites and aggregate it. The feeds (in XML format) are collected with an RSS reader. Readers for RSS include applications such as Mozilla Thunderbird and fancier readers such as Google Reader. Subscribers to RSS feeds maintain their privacy. How will RSS feeds be used at NGDC? Users will select RSS feeds for specific data types. Subscribers will get occasional automatic alerts for: format changes reprocessed data sets new data sets Coming soon!

Thank You! 15