1. HAO/SCD: VO, metadata, catalogs, ontologies, querying
Peter Fox, Jose Garcia, Patrick West (HAO)
Don Middleton, Luca Cinquini, Dave Brown (SCD)

2. OPeNDAP metadata mtg - Fox
Overview
Virtual Observatory and semantics
CEDARWEB
Earth System Grid
Metadata and catalog needs
Queries
Fundamental question - where does the system end and the user responsibility start? How is this different from where a framework ends - does it ever?
May 15-16, 2006
OPeNDAP metadata mtg - Fox

3. OPeNDAP metadata mtg - Fox
Recent definition
Workshop: A Virtual Observatory (VO) is a suite of software applications on a set of computers that allows users to uniformly find, access, and use resources (data, software, document, and image products and services using these) from a collection of distributed product repositories and service providers. A VO is a service that unites services and/or multiple repositories (from NASA VO workshop, October 2004)
VxOs - x is one discipline, you may have one and not call it a VO
Increasingly, the need is for VxyO - i.e. interdisciplinary (or VxyzO, etc.) and emphasizes integration (data, processes, etc.)
May 15-16, 2006
OPeNDAP metadata mtg - Fox

4. OPeNDAP metadata mtg - Fox
CEDARWEB 3.x
Data query, selection and retrieval interface, with integrated tools, e.g. ability to plot (preview) data before retrieving it.
To contrast the approach of proxy models with physics based models – no “free parameters” and absolute units
May 15-16, 2006
OPeNDAP metadata mtg - Fox

5. OPeNDAP metadata mtg - Fox
CEDARWEB 3.x
Ability to quickly plot data to assess suitability, quality, and produce a quick copy with some customization for a preliminary study.
May 15-16, 2006
OPeNDAP metadata mtg - Fox

6. Virtual Observatory: Need better glue
Basic problem: schema are categorized rather than developed from an object model/class hierarchy -> significantly limits non-human use. However, they all form the basis to organize catalog interfaces for all types of data, images, etc.
This limits data systems utilizing frameworks and prevents frameworks from truly interoperating (SOAP, WSDL only a start)
Directories, e.g. NASA GCMD, CEDAR catalog, FITS (flat) keyword/ value pairs, are being turned into ontologies (SWEET, VSTO)
Markup languages, e.g. ESML, SPDML, ESG/ncML are excellent bases
Evolve, recast, merge (where appropriate) using formal processes, tools with use-cases - for interface specifications, reasoning, validation, verification, inference, etc. beyond the usual search and access
May 15-16, 2006
OPeNDAP metadata mtg - Fox

7. OPeNDAP metadata mtg - Fox
t,X
May 15-16, 2006
OPeNDAP metadata mtg - Fox

8. Earth System Grid Portal
May 15-16, 2006
OPeNDAP metadata mtg - Fox

9. Metadata-centric view of ESG services
DATA TRANSPORT
USER AUTHENTICATION
AND AUTHORIZATION
LOCATION
METADATA
DATA ANALYSIS &
VISUALIZATION
ACCESS AND
AUTHORIZATION
METADATA
AGGREGATION
METADATA
METADATA
SERVICES
CATALOGUING
METADATA
CONTENT METADATA
DATA BROWSING
ANNOTATION & HISTORY
METADATA
LOGGING
METADATA
SYSTEM MONITORING
AND CONTROL
DATA SEARCH & DISCOVERY
May 15-16, 2006
OPeNDAP metadata mtg - Fox

10. OPeNDAP metadata mtg - Fox
ESG CLIENTS API
& USER INTERFACES
PUBLISHING
ANALYSIS & VISUALIZATION
SEARCH & DISCOVERY
ADMINISTRATION
BROWSING & DISPLAY
HIGH LEVEL METADATA SERVICES
METADATA
EXTRACTION
METADATA
ANNOTATION
METADATA & DATA
REGISTRATION
METADATA
BROWSING
METADATA
SEARCH, QUERY
& DISCOVERY
METADATA
AGGREGATION
METADATA
VALIDATION
METADATA
CONVERSION
METADATA
DISPLAY
CORE METADATA SERVICES
METADATA ACCESS
(update, insert, delete, query)
SERVICE TRANSLATION
LIBRARY
METADATA HOLDINGS
Replica
Location
Services
Metadata
Cataloguing
Services
THREDDS
catalogs
XML DB
May 15-16, 2006
OPeNDAP metadata mtg - Fox

11. OPeNDAP metadata mtg - Fox
Catalogs for ESG
Road analogy - users can choose their entry point (e.g. minor or major road) depending on need for the day/task.
May 15-16, 2006
OPeNDAP metadata mtg - Fox

12. Metadata and catalog needs
As we wanted to do more with the data we started adding more ‘use-metadata’ to the SQL catalog - was not scaling
Some queries that need to be asked OR some inference/ reasoning may depend upon
values of the data (as distinct from metadata, and/or
structure of the data, and/or
knowledge about the data and its related elements and/or
relation of this dataset/variable/attribute to another
Moving to Ontologies - need is to query the ontology: Xquery or SPARQL
Some of this can be done internally, some must call the SQL catalog (due to scaling/performance)
Not satisfying technologists needs ;-)
May 15-16, 2006
OPeNDAP metadata mtg - Fox

13. OPeNDAP metadata mtg - Fox
May 15-16, 2006
OPeNDAP metadata mtg - Fox

14. Virtual Observatories
Conceptual examples:
In-situ: Virtual measurements
Related measurements
Remote sensing: Virtual, integrative measurements
Data integration
May 15-16, 2006
OPeNDAP metadata mtg - Fox

15. OPeNDAP metadata mtg - Fox
The active Sun
May 15-16, 2006
OPeNDAP metadata mtg - Fox