1 ECHO and EDG Status Sept 12, 2006 Beth Weinstein, Yonsook Enloe,

2 Outline What is ECHO? ECHO Background - How did we get here? Why use ECHO? Current Functionality Current Partners Participation Upcoming activities

3 ECHO Mission/Vision Statement ECHO Mission ECHO’s mission is to enable a global marketplace of Earth Observation resources that will make Earth Observation data utilization more efficient and will spark innovation. ECHO provides Earth Observation communities with the ability to publish, discover, access and integrate directory and inventory level data and services through community-developed user interfaces. ECHO Vision ECHO will… be highly recognized, trusted and valued by the Earth Observation community be a critical building block in distributed information, modeling, decision support and public access systems have a low cost of participation to encourage broad community involvement

4 ECHO as Middleware ECHO is middleware for a service-oriented enterprise Its capabilities can be accessed through publicly available APIs Based on industry standards for performing web-based computing Defined in the Web Services Definition Language (WSDL) and are accessible through Simple Object Access Protocol (SOAP). Custom clients are developed to interact with ECHO’s Earth science metadata and extended services registry Clients written in most contemporary programming languages are isolated from the underlying technologies Clients may call the ECHO web services much like a local function call.

5 ECHO Background – How did we get here? ECHO initiated as an enhancement to EOSDIS in response to: User feedback on complexity and limitations of the “system-wide” view of EOSDIS data provided by EOS Data Gateway (EDG). Belief that the community could and would develop better client capabilities tailored to their needs. Evolving NASA Earth science vision of multiple, distributed, heterogeneous data and service providers. Availability of emerging technologies (e.g. web services). Response was development of ECHO as enabling infrastructure. “Externalized” metadata and made it accessible via APIs that supported development of custom clients. Extensible architecture that allows standard client and provider interfaces to be added. Support for data services. Centralized “clearinghouse” model based on industry feedback. Driven by performance and availability requirements.

6 Why Use ECHO? Open system provides Earth science data and services to large, diverse pool of users enabling scientific community interaction and collaboration Control in the hands of the data partner Automate mapping between your metadata and ECHO catalog metadata Control visibility and access to your contributed resources Select the best spatial search approach for your data Check the history of orders and provide status on open orders Users search for collection and inventory-level data Search and order data through a customized user interface Directly access online data and/or order data on media ECHO offers high system availability 99% system availability Even if your system is down, ECHO users can still search your metadata

7 ECHO Capability Today User Registration and Login Metadata ingest, validation, and reconciliation Searching is accurate and flexible - support for Cartesian, geodetic and orbital data description and subsequent search by these and the following mechanisms Spatial Search (e.g. line, polygon, multipolygon, circle) Temporal Range Searches Keyword Searches Numeric Searches (e.g. cloud cover percentage) Product Specific Attribute Searches Open interfaces (web service based) for human-machine or machine-to- machine clients (legacy interfaces are being phased out). Data Access Direct On-line Access Brokering of Orders Price Quotes Subscriptions Interoperability with other systems (OGC/NSDI Client support) Service Catalog based on web services standards

8 ECHO Schedule ECHO 7.0 operational Browse Data Insert, Update, and Deletes Multiple Collections and Groups for Access Control Rules Spatial Query Based on Lat/Lon Point Mar 2006 ECHO 8.0 operational (Phase out of XML APIs. Web Service APIs are primary interface mechanism.) Web Services API Asynchronous Queries Web Service brokering Oct 2006 ECHO 9.0 operational (Removal of XML APIs. Web Service APIs are only interface mechanism.) Improved performance More Comprehensive Error Handling Enhanced Security Metrics Apr 2007 ECHO 10.0 operational EOSDIS Evolution Items 4Q 2007

9 ECHO Data Partner Status ECHO’s Current Holdings (Aug 2006) from 10 Data Partners Collections2,408 Granules66 million Browse14 million All NASA ECS DAACs are actively participating in ECHO (GES, LARC, LP, NSIDC) Atmospheric Composition and Dynamics, Global Precipitation, Ocean Biology, Ocean Dynamics, Solar Irradiance Radiation Budget, Clouds, Aerosols and Tropospheric Chemistry Land Processes Snow and Ice, Cryosphere and Climate V0 DAACs are participating (ASF, GES, JPL, ORNL, SEDAC, PO) Synthetic Aperture Radar (SAR), Sea Ice, Polar Processes, Geophysics Biogeochemical Dynamics, Ecological Data, Environmental Processes Oceanic Processes, Air-Sea Interactions Population, Sustainability MODIS Data Processing System (MODAPS) is operational JAXA CEOP is in test mode

10 Data Provider Support in ECHO FTP Push for submitting metadata DTDs that govern required xml format available on echo website for metadata submission Required and optional Collection level and product (granule) metadata Browse metadata and imagery storage Automated processes for ingesting metadata files. Metadata update and reconciliation capabilities Partner test system to test metadata and perform validation before ingesting into operational system ECHO Ops team support for data providers

11 ECHO Client Partner Status Variety of client types Web-based clients Installed applications User interactive machine-to-machine Variety of purposes General purpose (e.g. Geospatial and temporal searching) Customized user interfaces (e.g. Targeted to smaller communities, Facilitates specific tasks) Back-end harvesting tools (e.g. Supports client-side caching of key information) Additional value-added processing by clients (e.g. Subset, Stitch, Resample, Reproject, Reformat) Middleware components NASA ROSES ACCESS solicitation client will be selected in Fall 2006.

12 Custom Clients and ECHO There are many different ways to use ECHO. Can use a subset of ECHO web service APIs to use only the desired ECHO functionality (search, order, access, extended services...) Can search through entire ECHO metadata or service registries or a subset Can search using full search criteria or limited search criteria or product specific attributes Can customize the search results returned to subset of data or service metadata or list of data access urls of result set Application-specific server can harvest ECHO metadata of interest and cache it for value added processing by its own dedicated client Clients can use ECHO Web Service APIs to register an extended service in the ECHO Service registry and access this service through SOAP Case example: OPeNDAP Group built a limited client access to ECHO Service Registry in less than one month. This client access was used by Matlab client and can be used by other OPeNDAP enabled clients. This client invokes a search that searches for only data served by OPeNDAP servers. Client queries by space & time, and is returned set of metadata & OPeNDAP urls of result set. Level of effort to build client to query ECHO using web services is very small.

13 EDG Transition to ECHO’s WIST Client Warehouse Inventory Search Tool (WIST) ECHO client being developed by NASA ESDIS General search and order interface Will offer all EDG functionality Public access to current ECHO operational version WIST releases Preoperational WIST has been deployed for public access. While it has been tested with ECHO, it is considered preoperational because not all of the EDG functionality has been included yet. Next version of WIST released in November 2006 will include subsetting and ASTER On Demand support. Operational WIST 1.0 will be available in April 2007 when ECHO Version 9.0 goes operational. ECHO Partners (e.g. NASA DAACs) will continue to test and evaluate WIST as new releases are made. EDG to WIST transition Goal is to start the EDG-to-WIST transition plan in summer 2007 and turn off EDG in late ECHO must meet criteria (e.g. search performance, available, up-to-date metadata) before EDG is turned off. EDG and WIST will run in parallel for a period of time.

14 International Partner/ECHO Activities CEOP (Coordinated Enhanced Observing Period) program Plans to use ECHO and OPeNDAP enabled clients and servers OPeNDAP enabled clients can directly query ECHO Satellite data used by CEOP will be represented in ECHO JAXA is currently evaluating ECHO through its CEOP activity Israel Space Agency will become an ECHO Data Partner. Also expect to be a client partner and service partner later. IRE RAS (Russia) will become an ECHO data partner, client partner, and service partner. ESA is implementing a prototype to access ECHO from INFEO through ECHO’s web services APIs. Dundee (Scotland) considering becoming an ECHO data partner and client partner. Continuing discussions with other international partners. Future discussions with GEO partners also expected. We welcome additional partners!

15 OPeNDAP/ECHO Activities CEOP program CEOP will ingest metadata repesenting data of interest (from JAXA, NASA, ESA, and Eumetsat) into ECHO. WTF-CEOP developing extensions to OPeNDAP based tools to provide access to satellite data to the CEOP science community. CEOP Satellite Data Server will provide access to satellite data served through a WCS to OPeNDAP enabled clients. OPeNDAP prototype Direct search and access of ECHO through the web service APIs by OPeNDAP clients Matlab client was demoed in July 2006 with operational capability expected when ECHO 8.0 is operational. This capability was demoed at the WTF-CEOP session yesterday.

16 ECHO & GEO Tasks ECHO can be a WGISS contribution to GEO – initial task list Task DA 06-06: Spatial Data Infrastructures Advocate use of existing Spatial Data Infrastructure components as institutional and technical precedents, where appropriate, including standard protocols and interoperable system interfaces, among other components. Task AR 06-05: GEO Clearinghouse Initiate development of a publicly accessible, network-distributed clearinghouse, subject to GEOSS interoperability specifications to date, and including an inventory of existing data, metadata, and pre-defined common products. Task WA In Situ Water Resource Monitoring Initiate the creation of a coordination mechanism within GEO for global in-situ water observations, including ocean observations, and advocate synergy and sharing of infrastructure among observing systems. Task WA Capacity Building Program in Latin America Task WA 07-P2 Satellite Water Measurements

17 IDN (GCMD)/ECHO Activities GCMD and ECHO are working together to share information from its registries and give users a more unified experience when interacting with the two systems GCMD Portal to ECHO data operational in 2Q 2007

18 ECHO Contact Information ECHO International contact– Yonsook Enloe Contact ECHO Operations (Ops) Visit the ECHO Project Website Holdings Summary, Upcoming Functionality, APIs and DTDs Reference materials and tools Real-time systems status monitor Operations metrics updated weekly Join ECHO Mailing List: Schedule bi-lateral telecons to discuss potential collaboration!