Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley National Center for Atmospheric Research, Boulder, CO *NOAA/CIRES, Boulder, CO AGU 2012 Fall Meeting: IN44A-05

2 Research Data Archive (RDA) at NCAR rda.ucar.edu distinct datasets for climate and weather research 2.Collections: ocean & atmosphere observations, analyses, reanalyses, operational NWP outputs 3.Free and open access AGU 2012 Fall Meeting: IN44A-05

ISPD Overview 3 World’s largest collection of surface & sea level pressure observations  Land station  Marine observations  Tropical cyclone best track Period (version 2): 1768 – 2010 Volume: 465 Gbyte Available since Aug 2010 AGU 2012 Fall Meeting: IN44A-05

ISPD Overview Contributors o Atmospheric Circulation Reconstructions over the Earth (ACRE) o Australian Bureau of Meteorology o British Antarctic Survey o Cook Islands Meteorological Service o Danish Meteorological Institute o Deutscher Wetterdienst (DWD; German Weather Service) o European and North Atlantic Daily to Multidecadal Climate Variability (EMULATE) o ETH Zurich, Switzerland o GCOS/WCRP Working Group on Observational Data Sets for Reanalysis o MANY MANY MORE…. Assembled by NOAA/ESRL, CIRES (Univ. of Colorado), & NOAA/NCDC AGU 2012 Fall Meeting: IN44A-05

20 th Century Reanalysis 5 Global reanalysis of atmospheric circulation Period: 1869 – 2010 Assimilates ISPDv2 as input Compo et al. (2011) QJRMS AGU 2012 Fall Meeting: IN44A-05

20 th Century Reanalysis: Oct 1950 mean 1000 hPa temperature 6  ISPD stations  ISPD marine obs AGU 2012 Fall Meeting: IN44A-05

7  ISPD stations  ISPD marine obs AGU 2012 Fall Meeting: IN44A th Century Reanalysis: Nov 1960 mean 1000 hPa temperature

8 ISPD obs assimilated into 20CR 20CR data quality control feedback contained in ISPD ­ Provides estimated uncertainty in obs ­ Helps improve underlying observational database AGU 2012 Fall Meeting: IN44A-05 ISPD & the 20 th Century Reanalysis

9 ISPD Sample Annual Station Distribution 1850 * Land stations only * No marine stations AGU 2012 Fall Meeting: IN44A-05

* Land stations only * No marine stations AGU 2012 Fall Meeting: IN44A-05 ISPD Sample Annual Station Distribution

* Land stations only * No marine stations AGU 2012 Fall Meeting: IN44A-05 ISPD Sample Annual Station Distribution

* Land stations only * No marine stations AGU 2012 Fall Meeting: IN44A-05 ISPD Sample Annual Station Distribution

13 ISPD Observations/Year Figure courtesy Chesley McColl, NOAA/ESRL 2010: 53 Million ~ 1.5 Billion total observations AGU 2012 Fall Meeting: IN44A-05

Data Access: Problem Background Large computational/storage resources needed –Store data –Extract desired data from large grids/files –Convert data to desirable format(s) 14 Scientific data centers have these resources Individual researchers generally don’t AGU 2012 Fall Meeting: IN44A-05

Goals –Make data more accessible and easier to use for individual researchers Reasonable access volumes Desired data formats User defined parameters/grids 15 Researchers stay focused on research AGU 2012 Fall Meeting: IN44A-05 Data Access: Problem Background

ISPD Data Access Services 16 Powerful computing NCAR Large disk storage (~ 0.5 PB) Rich and detailed metadata Direct file download via web Customized data sub-setting HDF-5 to ASCII software tools AGU 2012 Fall Meeting: IN44A-05

17 ISPD Metadata Features Both group- and file-level metadata Drive interfaces for file grouping and sub- setting tools Support efficient back-end processing Improve scalability Provide “quick look” at data samples AGU 2012 Fall Meeting: IN44A-05

ISPD Metadata Interface Example 18 AGU 2012 Fall Meeting: IN44A-05

19 AGU 2012 Fall Meeting: IN44A-05

20 AGU 2012 Fall Meeting: IN44A-05

21 AGU 2012 Fall Meeting: IN44A-05

Data Access: ISPD Subset Interface 22 AGU 2012 Fall Meeting: IN44A-05

ISPD Data Access Services 23 Temporal range sub- setting (daily) Spatial sub-setting Lat/Lon region Individual station ID

ISPD Data Access Services 24 Data sub-setting options (cont.) Observation type Land station Marine obs Radiosonde Dropsonde TC best track

ISPD Data Access Services 25 Subsetting processed in delayed mode notification Download via server-provided scripts (wget)

26 ISPD 2012 Subset metrics Data accessed: ~ 6.5 TB Data served: ~ 46 GB

Summary & Future Directions 27 RDA – Supply “User Friendly” Data  Parameter & spatial sub-setting  Metadata discovery  Format conversion  Improved and additional services NWSC-Cheyenne opening – more computing power AGU 2012 Fall Meeting: IN44A-05

28 DOI assignment Geoscience Data Journal article ISPD v3 ( )  Spring 2013 AGU 2012 Fall Meeting: IN44A-05 ISPD Forthcoming

29 rda.ucar.edu/ds132.0 AGU 2012 Fall Meeting: IN44A-05