HAO/SCD: VO, metadata, catalogs, ontologies, querying Peter Fox, Jose Garcia, Patrick West (HAO) Don Middleton, Luca Cinquini, Dave Brown (SCD)

OPeNDAP metadata mtg - Fox Overview Virtual Observatory and semantics CEDARWEB Earth System Grid Metadata and catalog needs Queries Fundamental question - where does the system end and the user responsibility start? How is this different from where a framework ends - does it ever? May 15-16, 2006 OPeNDAP metadata mtg - Fox

OPeNDAP metadata mtg - Fox Recent definition Workshop: A Virtual Observatory (VO) is a suite of software applications on a set of computers that allows users to uniformly find, access, and use resources (data, software, document, and image products and services using these) from a collection of distributed product repositories and service providers. A VO is a service that unites services and/or multiple repositories (from NASA VO workshop, October 2004) VxOs - x is one discipline, you may have one and not call it a VO Increasingly, the need is for VxyO - i.e. interdisciplinary (or VxyzO, etc.) and emphasizes integration (data, processes, etc.) May 15-16, 2006 OPeNDAP metadata mtg - Fox

OPeNDAP metadata mtg - Fox CEDARWEB 3.x Data query, selection and retrieval interface, with integrated tools, e.g. ability to plot (preview) data before retrieving it. To contrast the approach of proxy models with physics based models – no “free parameters” and absolute units May 15-16, 2006 OPeNDAP metadata mtg - Fox

OPeNDAP metadata mtg - Fox CEDARWEB 3.x Ability to quickly plot data to assess suitability, quality, and produce a quick copy with some customization for a preliminary study. May 15-16, 2006 OPeNDAP metadata mtg - Fox

Virtual Observatory: Need better glue Basic problem: schema are categorized rather than developed from an object model/class hierarchy -> significantly limits non-human use. However, they all form the basis to organize catalog interfaces for all types of data, images, etc. This limits data systems utilizing frameworks and prevents frameworks from truly interoperating (SOAP, WSDL only a start) Directories, e.g. NASA GCMD, CEDAR catalog, FITS (flat) keyword/ value pairs, are being turned into ontologies (SWEET, VSTO) Markup languages, e.g. ESML, SPDML, ESG/ncML are excellent bases Evolve, recast, merge (where appropriate) using formal processes, tools with use-cases - for interface specifications, reasoning, validation, verification, inference, etc. beyond the usual search and access May 15-16, 2006 OPeNDAP metadata mtg - Fox

OPeNDAP metadata mtg - Fox t,X May 15-16, 2006 OPeNDAP metadata mtg - Fox

Earth System Grid Portal May 15-16, 2006 OPeNDAP metadata mtg - Fox

Metadata-centric view of ESG services DATA TRANSPORT USER AUTHENTICATION AND AUTHORIZATION LOCATION METADATA DATA ANALYSIS & VISUALIZATION ACCESS AND AUTHORIZATION METADATA AGGREGATION METADATA METADATA SERVICES CATALOGUING METADATA CONTENT METADATA DATA BROWSING ANNOTATION & HISTORY METADATA LOGGING METADATA SYSTEM MONITORING AND CONTROL DATA SEARCH & DISCOVERY May 15-16, 2006 OPeNDAP metadata mtg - Fox

OPeNDAP metadata mtg - Fox ESG CLIENTS API & USER INTERFACES PUBLISHING ANALYSIS & VISUALIZATION SEARCH & DISCOVERY ADMINISTRATION BROWSING & DISPLAY HIGH LEVEL METADATA SERVICES METADATA EXTRACTION METADATA ANNOTATION METADATA & DATA REGISTRATION METADATA BROWSING METADATA SEARCH, QUERY & DISCOVERY METADATA AGGREGATION METADATA VALIDATION METADATA CONVERSION METADATA DISPLAY CORE METADATA SERVICES METADATA ACCESS (update, insert, delete, query) SERVICE TRANSLATION LIBRARY METADATA HOLDINGS Replica Location Services Metadata Cataloguing Services THREDDS catalogs XML DB May 15-16, 2006 OPeNDAP metadata mtg - Fox

OPeNDAP metadata mtg - Fox Catalogs for ESG Road analogy - users can choose their entry point (e.g. minor or major road) depending on need for the day/task. May 15-16, 2006 OPeNDAP metadata mtg - Fox

Metadata and catalog needs As we wanted to do more with the data we started adding more ‘use-metadata’ to the SQL catalog - was not scaling Some queries that need to be asked OR some inference/ reasoning may depend upon values of the data (as distinct from metadata, and/or structure of the data, and/or knowledge about the data and its related elements and/or relation of this dataset/variable/attribute to another Moving to Ontologies - need is to query the ontology: Xquery or SPARQL Some of this can be done internally, some must call the SQL catalog (due to scaling/performance) Not satisfying technologists needs ;-) May 15-16, 2006 OPeNDAP metadata mtg - Fox

OPeNDAP metadata mtg - Fox May 15-16, 2006 OPeNDAP metadata mtg - Fox

Virtual Observatories Conceptual examples: In-situ: Virtual measurements Related measurements Remote sensing: Virtual, integrative measurements Data integration May 15-16, 2006 OPeNDAP metadata mtg - Fox

OPeNDAP metadata mtg - Fox The active Sun May 15-16, 2006 OPeNDAP metadata mtg - Fox