THREDDS Catalogs Ethan Davis UCAR/Unidata NASA ESDSWG Standards Process Group meeting, 17 July 2007

THREDDS Catalogs XML documents over HTTP Named datasets with access URLs (typed for various protocols) Datasets can be hierarchically organized (good for browsing) Metadata at collection (inheritable) or atomic dataset level.

THREDDS Catalogs

Catalog Services Catalog Validation Catalog Subsetting HTML generation Automatic/dynamic catalog generation and dataset metadata harvesting Direct metadata harvesting OAI-PMH harvesting (DIF, ADN)

Implementations – Servers THREDDS Data Server (TDS) OPeNDAP Hyrax server GrADS Data Server (GDS) IRI/LDEO server (Ingrid) NCAR Community Data Portal (CDP)

Implementations - Clients IDV MyWorld GIS netCDF-Java ToolsUI NCAR Community Data Portal (CDP) THREDDS Data Server (TDS)

Installations NCAR/UCAR NOAA – NCDC, NGDC, PMEL, … COLA FNMOC BOM (Aust) GMU IRI/LDEO …

Leader For SPG Endorsement Process ??? User community member?

CF (Climate and Forecast) Metadata Convention for netCDF Ethan Davis UCAR/Unidata NASA ESDSWG Standards Process Group meeting, 17 July 2007

CF Design Goals “The Climate and Forecast (CF) metadata conventions are designed to promote the processing and sharing of data stored in files created with the netCDF API. Fundamental features of the conventions are that CF-aware software can automatically determine the space-time location of variables (facilitating analysis and graphical display), and metadata describing each variable is sufficiently detailed to determine whether variables from different sources are comparable.” -- Lawrence, B.N., R. Drach, et al, “Maintaining and Advancing the CF Standard for Earth System Science Community Data”.

CF Guiding Principles 1.Data should be self-describing. No external tables are needed to interpret the file. For instance, CF encodings do not depend upon numeric codes (by contrast with GRIB). 2.Conventions have been developed only for known issues. Instead of trying to foresee the future, features are added as required. 3.The convention should be easy to use, both for data-writers and users of data. 4.The metadata and the semantic meaning encoded through the metadata should be readable by humans as well as easily utilized by programs. 5.Redundancy should be minimised as much as possible (so as to reduce the chance of errors of inconsistency when writing data). -- Gregory (2003), (broken link)

CF netCDF Metadata Conventions Determine if variables are comparable: –standard names –units Locate in space and time: –Coordinate variables –Projections –etc Data types (gridded,...)

CF, not just for netCDF anymore? CF specification defined for netCDF-3 But much of CF is format agnostic, e.g., CF-style metadata has been added to HDF data The netCDF-4 data model has significant changes from the netCDF-3 data model. Won’t be sure how netCDF-4 will affect CF until we have more experience with it

CF Community Initially dominated by climate and forecast modelers Community is broadening: oceanography, atmospheric chemistry, biology,... Governance: Conventions Committee, Standard Name Committee

Implementations CF compliance checker libcf (support read/write of CF compliant netCDF) netCDF-Java (read, some support for write) ArcGIS 9.2+ reads CF-netCDF files CF-netCDF being developed as OGC WCS data format profile CF-netCDF as TDS netCDF Subset Service response

Leader For SPG Endorsement Process User community member Governing committee member?