GALEON IE: data model reconciliation between netCDF and WCS/GML Stefano Nativi – CNR-IMAA and University of Florence (Italy) Lorenzo Bigagli – CNR-IMAA and University of Florence (Italy) Ben Domenico – Unidata/UCAR (USA) John Caron – Unidata/UCAR (USA)

Outline Rationale Data models Reconciliation approach Available implementation: the ncML-GML Examples

Rationale We must explicitly mediate from netCDF hyperspatial data to WCS coverage models netCDF datasetWCS coverage N independent dimensions (i.e. axes)2, 3, 4 coverage domain dimensions Set of scalar variablesCoverage range-set of values (t, z, y, x) variable shape(x, y, z, t) domain shape Implicit geo-location metadataExplicit geo-location metadata Grid geometry irregularly spacedGrid geometry regularly spaced etc.

NetCDF-CF dataset content

WCS coverage content

2 Dimension Coordinate System Implicit/explicit Geometry Range set Spatial Reference System (SRS) 2 Dimension Coordinate System Implicit/explicit Geometry Range set Spatial Reference System (SRS) 2 Dimension Coordinate System Implicit/explicit Geometry Range set Spatial Reference System (SRS) 2 Dimension Coordinate System Implicit/explicit Geometry Range set Spatial Reference System (SRS) Mediation Process netCDF hyperspace dataset (3/4/5D) 2D + elev + time Coverages 2D+elev+time dataset 2D SCS + elev + time Implicit/explicit Geometry Range set Spatial Reference System (SRS) a Coverage … … N-Dimension Coordinate Systems explicit/semi-implicit/implicit Geometry Scalar measured quantities

Conceptual mapping NetCDF conceptsMapping cardinality Geo-Information concepts Dataset1…nCoverage Dimensionn…mGrid/Multipoint Domain, CS, CRS Variablen…mScalar/parametric Rangeset, Grid/Multipoint Domain, CS, CRS Attributen…mAny Semantics level

Mediation process: the introduced GI concepts in brief A netCDF dataset origins several different coverages Each coverage is characterized by a domain, a range- set and is referenced by a CS/CRS Each coverage is optionally described by a geographic extent Each domain is characterized by a geometry –Supported domains: regular grid domain, irregular grid domain and multipoint domain Each range-set lists or points set of values associated to each domain location –Supported range-set types: scalar range-set and parametric range-set

An Implementation already exists The ncML-GML markup language implements the presented reconciliation model It is a Mediation Markup Language between ncML (netCDF Markup Language) and GML –An extension of ncML core schema, based on GML grammar NcML-GML version –based on GML N2G version 0.8 –Java API for ncML-GML ver WCS-G –WCS 1.0 which supports ncML-GML/netCDF documents Subsetting (domain and range-set) –netCDF –ncML-GML WCS light client –Test client for WCS-G

Examples

Example: sst.nc Example: sst.nc (~3Mb) –Sea Surface Temperature (float) depending on lon = 180 ; lat = 170 ; time = UNLIMITED ; // (24 currently) –Already published on OPeNDAP server

netCDF-CF Unrestricted domain Special semantics attached to dimension: –Latitude –Longitude –Vertical –Time –Pressure (added by us) Scalar co-domain only Recommended axes order – *-T-Z-Y-X

sst.nc – netCDF-CF annotated CDL netcdf C:/WebDocs/THREDDS/GALEON/SST.NC.test/sst-again.nc { dimensions: lon = 180; // (has coord.var) lat = 170; // (has coord.var) time = UNLIMITED; // (24 currently) // (has coord.var) bnds = 2; variables: double lon(lon); :standard_name = "longitude"; :long_name = "longitude"; :units = "degrees_east"; :axis = "X"; :bounds = "lon_bnds"; :original_units = "degrees_east"; double lon_bnds(lon, bnds); (continue) CF assign special semantics to some dimensions CF mandates the use of coordinate variables for such dimensions Semantics is associated based on :units attribute CF conventions support partitioning an axis in cells Standard names are defined for common dimensions/varia bles

double lat(lat); :standard_name = "latitude"; :long_name = "latitude"; :units = "degrees_north"; :axis = "Y"; :bounds = "lat_bnds"; :original_units = "degrees_north"; double lat_bnds(lat, bnds); double time(time); :standard_name = "time"; :long_name = "time"; :units = "days since "; :axis = "T"; :calendar = "360_day"; :bounds = "time_bnds"; :original_units = "seconds since "; double time_bnds(time, bnds); (continue) Semantics is associated based on :units attribute

float tos(time, lat, lon); :standard_name = "sea_surface_temperature"; :long_name = "Sea Surface Temperature"; :units = "K"; :cell_methods = "time: mean (interval: 30 minutes)"; :_FillValue = 1.0E20; // float :missing_value = 1.0E20; // float :original_name = "sosstsst"; :original_units = "degC"; :history = " At 16:37:23 on 01/11/2005: CMOR altered the data in the following ways: added E+02 to yield output units; Cyclical dimension was output starting at a different lon;"; (continue) CF conventions support the description of methods applied to obtain interval values

:title = "IPSL model output prepared for IPCC Fourth Assessment SRES A2 experiment"; :institution = "IPSL (Institut Pierre Simon Laplace, Paris, France)"; :source = "IPSL-CM4_v1 (2003) : atmosphere : LMDZ (IPSL-CM4_IPCC, 96x71x19) ; ocean ORCA2 (ipsl_cm4_v1_8, 2x2L31); sea ice LIM (ipsl_cm4_v"; :contact = "Sebastien Denvil, :project_id = "IPCC Fourth Assessment"; :table_id = "Table O1 (13 November 2004)"; :experiment_id = "SRES A2 experiment"; :realization = 1; // int :cmor_version = 0.96; // float :Conventions = "CF-1.0"; :history = "YYYY/MM/JJ: data generated; YYYY/MM/JJ+1 data transformed At 16:37:23 on 01/11/2005, CMOR rewrote data to comply with CF standards and IPCC Fourth Assessment requirements"; :references = "Dufresne et al, Journal of Climate, 2015, vol XX, p 136"; :comment = "Test drive"; Mandatory global attribute for CF conventions

Dataset vs. Coverage Generic mapping issue D(*-T-Z-Y-X) R C(X-Y-Z-T) *

Example: Lat-Lon-Time Lon Time Lat <GML 2D+T scalar coverage> netCDF file transcoding netCDF ncML-GML WCS

Example: 2 x Lat-Lon-Time Lon <GML 2D+T composite coverage> transcoding netCDF ncML-GML WCS Time Lat Lon Time Lat netCDF file

Example: Pressure-Lat-Lon-Time Lon Time Lat <GML 2D+T parametric coverage> netCDF file transcoding netCDF ncML-GML WCS P

netCDF Example: Lat-Lon-Time + Pressure-Lat-Lon-Time <GML 2D+T parametric coverage> transcoding netCDF WCS-G X Time Y P X Y netCDF file <GML 2D+T scalar coverage> ncML-GML