® OGC Web Services Initiative, Phase 9 (OWS-9): Innovations Thread - OPeNDAP James Gallagher and Nathan Potter, OPeNDAP © 2012 Open Geospatial Consortium.

Slides:



Advertisements
Similar presentations
1 NASA CEOP Status & Demo CEOS WGISS-25 Sanya, China February 27, 2008 Yonsook Enloe.
Advertisements

The Next Generation Network Enabled Weather (NNEW) SWIM Application Asia/Pacific AMHS/SWIM Workshop Chaing Mai, Thailand March 5-7, 2012 Tom McParland,
James Gallagher OPeNDAP 1/10/14
Chapter 10: Designing Databases
OPeNDAP in the Cloud Optimizing the Use of Storage Systems Provided by Cloud Computing Environments OPeNDAP James Gallagher, Nathan Potter and NOAA/NODC.
OPeNDAP’s Server4 Building a High Performance Data Server for the DAP Using Existing Software Building a High Performance Data Server for the DAP Using.
Streaming NetCDF John Caron July What does NetCDF do for you? Data Storage: machine-, OS-, compiler-independent Standard API (Application Programming.
Best Practices to Promote Data Interoperability Chris Lynnes Joe Glassy Technology Infusion Working Group.
Implementation of a Data Node in China's Spatial Information Grid Based on NWGISS Dengrong Zhang, Le Yu, Liping Di Institute of Spatial.
Planned Title: Review of Evaluation of Geospatial Search Allan Doyle.
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
OPeNDAP Present and Future An Overview Encompassing Current Projects & Potential New Directions Dave Fulker and James Gallagher.
Citation and Recognition of contributions using Semantic Provenance Knowledge Captured in the OPeNDAP Software Framework Patrick West 1
Unidata TDS Workshop THREDDS Data Server Overview October 2014.
Managing Data Interoperability with FME Tony Kent Applications Engineer IMGS.
THREDDS Data Server, OGC WCS, CRS, and CF Ethan Davis UCAR Unidata 2008 GO-ESSP, Seattle.
ToolMatch: Discovering What Tools can be used to Access, Manipulate, Transform, and Visualize Data Patrick West 1 Nancy Hoebelheinrich.
OPeNDAP and the Data Access Protocol (DAP) Original version by Dave Fulker.
Implementation of Model Data Interoperability for IOOS: Successes and Lessons Learned Rich Signell USGS Woods Hole, MA / NOAA Silver Spring USA Model Data.
, Increasing Discoverability and Accessibility of NASA Atmospheric Science Data Center (ASDC) Data Products with GIS Technology ASDC Introduction The Atmospheric.
Bringing it All Together: NODC’s Geoportal Server as an Integration Tool for Interoperable Data Services Kenneth S. Casey, Ph.D. YuanJie Li NOAA National.
, Implementing GIS for Expanded Data Accessibility and Discoverability ASDC Introduction The Atmospheric Science Data Center (ASDC) at NASA Langley Research.
Unidata’s TDS Workshop TDS Overview – Part II October 2012.
Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.
THREDDS Data Server Ethan Davis GEOSS Climate Workshop 23 September 2011.
OPeNDAP Developer’s Workshop Feb OPeNDAP 4 Data Server – Hyrax James Gallagher and Nathan Potter 21 Feb 2007.
Coverages and the DAP2 Data Model James Gallagher.
Weathertop Consulting, LLC Wednesday, January 14, 2009 IIPS 11A.2 1 A General Purpose System for Server-side Analysis of Earth Science Data Roland Schweitzer.
Mid-Course Review: NetCDF in the Current Proposal Period Russ Rew
Tools in Support of a National DMAC Derrick Snowden NERACOOS/ODP Annual Meeting 26 Sep 2012.
DELIVERING ENVIRONMENTAL WEB SERVICES (DEWS) Partners: UK Met Office (Lead Partner), British Atmospheric Data Centre (BADC), British Maritime Technology.
Accomplishments and Remaining Challenges: THREDDS Data Server and Common Data Model Ethan Davis Unidata Policy Committee Meeting May 2011.
The netCDF-4 data model and format Russ Rew, UCAR Unidata NetCDF Workshop 25 October 2012.
1 International Standards for Data Interoperability GALEON Geo-interface for Air, Environment, Land, Ocean NetCDF Ben Domenico Unidata Program Center*
Integrating netCDF and OPeNDAP (The DrNO Project) Dr. Dennis Heimbigner Unidata Go-ESSP Workshop Seattle, WA, Sept
DAP4 James Gallagher & Ethan Davis OPeNDAP and Unidata.
Documentation from NcML to ISO Ted Habermann, NOAA NESDIS NGDC.
Unidata TDS Workshop THREDDS Data Server Overview
Citation and Recognition of contributions using Semantic Provenance Knowledge Captured in the OPeNDAP Software Framework Patrick West 1
Unidata’s Common Data Model and the THREDDS Data Server John Caron Unidata/UCAR, Boulder CO Jan 6, 2006 ESIP Winter 2006.
PREMIS Implementation Fair, San Francisco, CA October 7, Stanford Digital Repository PREMIS & Geospatial Resources Nancy J. Hoebelheinrich Knowledge.
THREDDS Catalogs Ethan Davis UCAR/Unidata NASA ESDSWG Standards Process Group meeting, 17 July 2007.
May 2003National Coastal Data Development Center Brief Introduction Two components Data Exchange Infrastructure (DEI) Spatial Data Model (SDM) Together,
Page 1 CSISS Center for Spatial Information Science and Systems Access HDF-EOS data with OGC Web Coverage Service - Earth Observation Application Profile.
Unidata’s TDS Workshop TDS Overview – Part I July 2011.
HDF4 OPeNDAP Project Progress Report MuQun Yang and Hyo-Kyung Lee 1 HDF Developers' Meeting11/24/2015.
Remote Data Access with OPeNDAP Dr. Dennis Heimbigner Unidata netCDF Workshop October 25, 2012.
1 NASA CEOP Final Summary CEOS WGISS-26 Boulder, Colorado September 23, 2008 Yonsook Enloe
1 MESSAGE EXCHANGE FOR Web Service-Based Mapping Services AHMET SAYAR INDIANA UNIVERSITY COMMUNITY GRIDS LAB. COMPUTER SCIENCE DEPARTMENT August 17, 2005.
Distributed Data Analysis & Dissemination System (D-DADS ) Special Interest Group on Data Integration June 2000.
Weathertop Consulting, LLC Server-side OPeNDAP Analysis – Concrete steps toward a generalized framework via a reference implementation using F-TDS Roland.
LAS and THREDDS: Partners for Education Roland Schweitzer Steve Hankin Jonathan Callahan Joe Mclean Kevin O’Brien Ansley Manke Yonghua Wei.
OPeNDAP Developer’s Workshop Feb Server-side Functions for Geo-spatial Selection James Gallagher 22 Feb 2007.
OPeNDAP’s Server4: Building a High Performance Data Server for the DAP Using Existing Software James Gallagher*, Nathan Potter*, Patrick West**, Jose Garcia**
ESIP Air Quality Jan Air Quality Cluster Air Quality Cluster Technology Track Earth Science Information Partners Partners NASA NOAA EPA (?) USGS.
Copyright 2007, Information Builders. Slide 1 iWay Web Services and WebFOCUS Consumption Michael Florkowski Information Builders.
1 Key Results from GALEON John Caron Ben Dominico UCAR/Unidata.
OGC Web Services with complex data Stephen Pascoe How OGC Web Services relate to GML Application Schema.
1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)
Update on Unidata Technologies for Data Access Russ Rew
James Gallagher, Nathan Potter and NOAA/NODC
Using Community-Driven Metadata Conventions with On-Line Data
Product Designer Hub – Taking HPD to the Web
Data Browsing/Mining/Metadata
Accessing Remote Datasets through the netCDF interface.
Session 2: Metadata and Catalogues
Future Development Plans
OPeNDAP’s Server4: Building a High Performance Data Server for the DAP
OPeNDAP/Hyrax Interfaces
Adapting an existing web server to S3
Presentation transcript:

® OGC Web Services Initiative, Phase 9 (OWS-9): Innovations Thread - OPeNDAP James Gallagher and Nathan Potter, OPeNDAP © 2012 Open Geospatial Consortium

Goals Compare WCS 2.0 and DAP 2.0 Investigate using OPeNDAP’s Hyrax data server as a backend for WCS 2.0 by… evaluate the effort in upgrading an existing server from WCS 1.x to 2.0 evaluating the complexity of supporting several of the formats commonly used with GIS systems.

WCS and DAP: data models DAP (Data Access Protocol) supports a domain- neutral data model, both for data and metadata This enables it to serve as a transport for many kinds of information and… data that use many different metadata standards/conventions. Subsetting takes place using common programming operations like choosing array slices, structure fields, and so on

… data models WCS uses a domain-aware data model and interface It provides an integrated way to describe (metadata) coverages (data) and… subset them in the geospatial and temporal domains It also provides range subsetting operations

Hyrax (DAP) as a backend for WCS A DAP server (e.g., Hyrax, TDS) makes a good backend for WCS It provides a subsetting interface that abstracts many different ad hoc data stores The WCS service can focus on mapping between the WCS data model and the DAP data model The DAP server can provide uniform access to many different data formats

Does the DAP data model provide useful abstractions for a WCS service? Adding support for new response formats was straightforward. Reading new data (source) formats was similarly easy Since each format was accessed uniformly using DAP, the WCS service required no change for the new source formats and minimal changes for the new response formats.

Modular BES Processing Plans

What was coded in the WCS Service? The metadata information in the data sources is not abstracted by DAP, instead it is adapted to a common representation but the content is not altered at all. The WCS service had to be coded for the specific metadata standards and conventions used by the underlying data sources

WCS 1.x  2.0 Substantially different XML meant that we had to rework the software that (automatically) builds WCS XML The WCS 2.0 XML schema set is much easier to use than the 1.x XML; it validates and is better organized Heavy reliance on GML does not reduce the number of distinct specifications that must be read, however!

Issues in serving science data Phrases like ‘Science data’ oversimplify… However, many scientific data sets for multi- instrument platforms use complex hierarchical formats like HDF5 or NetCDF4 Files in these formats often hold tens or hundreds of variables Meanwhile, widely-supported WCS response formats are typically limited to a single or small number of ‘variables’ (aka ‘fields’)

…and not all science data is georeferenced and projected This is true even for ‘Earth science data’ In OWS-9 NASA provided a collection of data sets that were in ‘satellite’ coordinates and had to be projected by the WCS. Note that WCS supports coverages that could be used to represent these data, but clients would be hard pressed to make use of them.

Summary WCS 2.0 and DAP 2.0 differ in some key ways that make them complementary DAP servers provide good interfaces to data for WCS service implementations In this project, support for new data formats was abstracted almost completely by the DAP server However, metadata standards/conventions, however, were not abstracted by DAP Some response formats for WCS require complex transformations for some kinds of data