ECHO For International Partners March 8, 2005 Robin Pfister Yonsook Enloe

EDG to WIST Transition Plan V0 system first went operational in NASA ESDIS moving to next generation system – ECHO. EDG client currently provides search and order capabilities to V0 servers and to the ECS servers through the V0 interface Multiple CEOS partners host V0 servers or a translator/bridge that speaks V0 to/from their agency inventory systems NASA DAACs currently analyzing whether to support both ECHO and V0 interface or support just the ECHO system. Each DAAC will make an independent decision based on resources. Decisions will come late 2005 or early WIST (Warehouse Inventory Search Tool) client which provides comprehensive inventory search for ECHO will start operations in US EDG clients will operate through End date for US EDG clients is Jan 31, After US EDG clients are disabled, a final V0 Valids run for international sites who still use EDG will be made but with the US V0 Servers removed. No further maintenance will be performed to update the V0 valids file.

What Is ECHO? Functionally  Registry of spatial, temporal, numeric and textual metadata  Order broker  User and provider account service  Services registry and broker (future) Objectives  ECHO is a portal to Earth Science data and services.  It allows providers of data to share their metadata and offload some of their search responsibilities.  It also brokers orders from clients to the appropriate providers, providing tracking services for both the client and the provider.  ECHO presents a messaging interface based on XML, but does not currently provide a search GUI.

System Drivers Present an API for organizations to connect their own user interfaces and programs to Make it easy for providers of Earth Science data and services to participate in the system Provide searches that respond quickly Broker orders for both data and services Minimize operational costs Build upon advances in industry and use e- commerce systems as a model Build a system that can be scaled up to handle large numbers of requests Architecture Overview:

Context Community Tailored View of ECHO holdings Application Metadata and Service Registries Service Service with a UI Data Inventory KEY: Community Non-ECHO Registry ECHO-based Registry ECHO Registry XML Messages XML Message via Adaptors Anyone can participate From any level Provider-set access controls

Approach: Layered Architecture BMGT Proxy Ingest (XML via FTP) Full Record Insert, Update, Delete Partial Record Insert, Update, Delete Client OGC Client OGC Adapter SOAP Adapter Client Interface (RMI) (ECHO API Services) Partner Mgmt Catalog Search Place Order User Accts Admin API Config Info ECHO Kernel Business Objects Browse Image Repository Data Registry Service Registry Interoperable Quote Request Cancel Request Submit Request Partner Interface (SOAP) (Quote, Submit, Cancel) V0 GW Proxy ORNL Proxy

Approach: Interoperable Registries Service Registry Service offerings leveraging Web Service Standards Publish Capability  Advertisements  Service Interfaces  Service Implementations  Service GUIs  Linkage to Data Registry Discovery Capability  UDDI Standard Based  Data Registry View What services are appropriate for this collection/granules? Future  Service Brokering (ECHO 8.0)  Orchestration of Service Chains  Access Controls Data Registry Representing data resources through metadata Publish Capability Supports Collection, Granules and Browse publication Product Specific Attributes extend the standard data model Discovery Capability Collection/Dataset Granule/Inventory Based on Z39.50 Standard Full Results Management Access Capability Online Access Legacy Order Mechanisms Access Controls Visibility of data resources Access to data resources Metadata Subscriptions In the future will enable orchestration of Data and Service brokering and chaining

Partner Status Data Partners:  5 Operational Data Providers  5 Development/Test  Nearly 42 Million data granules (including those in ECS’ Data Pools) and over 7 Million Browse Images available from ASF, GSFC-ECS, LPDAAC-ECS, NSIDC- ECS, ORNL DAAC, SEDAC, Stennis (other providers in test or planning).  ECHO holdings report available from the echo website (eos.nasa.gov/echo), click on “status”, click on “operations reporting”. Client Partners:  3 Operational  11 Development/Test

ECHO Client Partners Operational  Mercury EOS – ORNL client  Power User Interface – script for bulk ordering  SIMECC (Simple MODIS ECHO Client) – MODIS data search and order In Test  WIST – EOS Data Gateway using ECHO  ASF ECHO Client - client built by the Alaska SAR Facility  WISRD (Web Interface for Searching, Subsetting, Stitching, Resampling, Regriding, and Reformating Data) – NSIDC Client to search for swath, scene, and gridded data  MODIS Website – client via existing MODIS website and ORNL’s shopping cart  Data Validation User Interface – client for MODIS Land Data Validation Team Under development  Annoterra Version2 – link between EOS newsfeeds, GCMD, and ECHO granules  SNOWI-E – NSIDC Client  NEO  Invasive Species Data Service There are plans for other clients (MODIS rapid response, OGC client)

Availability ECHO uses a cluster and load balancing approach to address availability We are in the process of maturing to a completely redundant system planned by the end of 2005 The system is monitored by automated systems 24X7, with pager notification of current, and in some cases pending, faults

ECHO Website Tour

ECHO Data Partner Participation

International Participation in ECHO Active data provider Inactive data provider by allowing ECHO to harvest metadata and store it in the ECHO Registry (ECHO needs to complete development of a harvester) Become a distributed search provider where ECHO issues a distributed search to the external system (ECHO future plans):  Distributed search to a separate ECHO Registry  Distributed search to a separate non-ECHO system

Active Data Provider Options Download, validate, and correct metadata periodically to ECHO Search & Order options :  Keep own agency data system unchanged and do not provide access to NASA data  Or keep own agency data system and provide access to NASA data through ECHO Client APIs.  Or discontinue agency data system for user access but develop own agency client to search desired subset of ECHO metadata & services, using ECHO Client APIs and some reusable software and software libraries

Inactive Data Provider Options ECHO project planning on developing metadata harvester in the near future (1 to 2 years?) Data Provider provides information for ECHO harvester ECHO harvests metadata periodically Search & Order Options :  Data provider keeps own agency system  If access to NASA data is desired, agency develops capability of agency client to access ECHO through established ECHO Client APIs.

ECHO Distributed Search Data Provider Use open source ECHO software to install and populate ECHO Registry at agency. The content of the agency ECHO Registry will be distinct from the content of the NASA ECHO Registry Become ECHO distributed search data provider using non-ECHO system  Will need to agree on search distribution interface and common metadata search & present attributes with ECHO team. Can be bi-directional interface.  NASA ECHO will also respond to search requests through already established ECHO Client APIs. Can CEOS international community desiring this option agree on one comment interface?

Future ICS Happenings with ECHO ECHO data model mapping to CIP and to ISO ECHO Client APIs comparison to CIP and to OGC Catalog 2.0 (general model & http profile) The details of these analyses will be discussed at future ICS telecons. We promise to focus the ECHO discussion at the ICS telecons by breaking up topics into manageable chunks. No marathon telecons! Schedule bi-lateral telecons with agency partners desiring ECHO information

Provider Account New ECHO Data Partners fill out an application to become a Data Partner  Data Partners “Getting Started” on the website,  ECHO Operations (Ops) at An ECHO Provider Account is created  Can associate multiple ECHO Registered User Accounts to manage the Provider Account  Maintains a list of named contacts (i.e. Order Manager, User Services, etc.)  Providers can establish lists of users and give them privileged access to provider metadata and ordering privileges

Next Steps: ECHO Development Highly-available, redundant back-up facility Service Brokering Asynchronous Searching Reference client Event Subscriptions Orchestration/Service Chaining Interagency and International Interoperability

ECHO Operations The ECHO Operations Team (ECHO Ops) is the point of contact for direct interaction between ECHO, its Partners, and end users ECHO Ops is responsible for the operation and maintenance of the ECHO operational and partner test systems ECHO Ops general support for Partners includes:  Assisting Partners in understanding the ECHO API and DTDs  Ingest management and accounting  Problem tracking and resolution  Advertising Data Partner holdings and availability of new datasets  Promoting and engaging new Client Partners

Support: Tools, Training, Documentation ECHO Ops  Staff Availability: Sysadmin 24X7, Ops Staff 11X5  ECHO website   User’s Guide, Holdings Summary, Upcoming Functionality, APIs and DTDs, Training Materials organized by partner type ECHO mailing lists  echo-all, echo-status, echo-client, echo-data ECHO Technical Committee (ETC) Meetings  All partners can participate in weekly telecons to share and discuss approaches, issues, solutions.  Contact ECHO Ops for dial-in information