1. ECHO Browse Reclassification Document ID: ECHO_Ops_Con_023 Version: 2

2. Presentation Outline Multi-Format Ingest Current ECHO Data Model Current ECHO Browse Handling Motivation for Change Proposed ECHO Data Model Proposed ECHO Browse Handling Sample Metadata Modifications System Impacts Questions 2

3. Multi-Format Ingest Multi-Format Ingest Workflow – FTP Delivery 1.Data Provider submits ECHO 10 formatted metadata to ECHO ftp polling location 2.FTP Ingest Service processes metadata and inserts into relational ECHO Database 3.Metadata Crawler reconstitutes ECHO 10 formatted metadata & inserts via REST API 4.REST API Inserts metadata in XML storage database for subsequent retrieval and indexing. – REST API Delivery 1.Data Provider submits ECHO 10 formatted to ECHO Ingest REST API 2.REST API Inserts metadata in XML storage database for subsequent retrieval and indexing.

4. Current ECHO Data Model Collection Granule Browse ECHO- Hosted Image Required Link Optional Link DAAC- Hosted Image

5. Current ECHO Browse Handing Browse Associations – Collection and Granule metadata records include a reference to an associated browse image but do not have any metadata for that browse record. – Separate browse metadata records contain all relevant metadata and do not include any reference to an associated collection or granule record. Ingest vs. Results – ECHO Ingest requires that a provider submit browse information in a separate metadata record apart from the associated collection or granule. However, the current DTD query results insert browse metadata inline with the collection and granule metadata returned to a user. ECHO vs. Externally Hosted Browse – ECHO Data Providers may choose to export the binary browse image files to be hosted by ECHO. In this case, the associated browse metadata specifies the browse file name and size. – ECHO Data Providers may also choose to export only the browse metadata with the URL of an externally hosted browse image. Browse Record Management – When a collection or granule that has a browse association is deleted, the linkage to the browse is deleted, but the browse record remains in ECHO. Providers must manage their browse record holdings in ECHO separate from their collection and granule records.

6. Motivation for Change Internal Data Model Inconsistency – ECHO data providers export browse metadata records along with their collections and granules. However, when a user or ECHO client queries for metadata, the ECHO results DTDs collapse inject the browse metadata into the collection and granule results. We are already removing the concept of a separate browse record during query & results presentation, creating an inconsistency in our data model. Format Translation Inconsistency – Formats, such as ISO 19115, do not have a separate browse record. Translating between ECHO and ISO 19115 is made unnecessarily complicated when dealing with browse record associations. Removing the separate browse record would create a more consistent pattern compared to other external formats. Simplified Data Exports – Removing the separate browse record would simplify ECHO data provider exports. Data providers would only send collection and granule records, dropping the browse and potentially the partial update formats. Simplified Ingest REST API – There is increased complexity supporting browse retrieval and formatting through the REST API that could be avoided without a separate browse record. Simplified Query REST API – ECHO Clients utilizing the new ECHO REST API would be forced to request collection or granule metadata and the separate browse metadata when working with ECHO 10 metadata, but not with the ISO 19115 or Atom+Frost formats. Removing the separate browse record will simplify API interactions.

7. Proposed ECHO Data Model Required Link Optional Link Collection Granule Browse ECHO- Hosted Image DAAC- Hosted Image

8. Proposed ECHO Browse Handling Browse Reclassification – The ECHO Collection and Granule metadata formats will be enhanced with a new metadata element containing all relevant information for an associated browse image. – The ECHO Browse metadata format would exist only for legacy Ingest support, but would not be made available to end users. – The associated browse image metadata reference element in the Collection and Granule record would persist in the schema only for legacy Ingest support, but would not be available to end users. Multi-Format Ingest Metadata – The Multi-Format Ingest metadata crawlers will translate collections and granules using the updated schema that utilizes the new elements. – The format Browse record metadata will not be supported for Ingest via the REST API ECHO vs. Externally Hosted Browse – There are no changes to how browse images are hosted within ECHO associated with this proposal. Browse Record Management – Data partners utilizing the legacy FTP Ingest will continue browse image management as is. – Data partners utilizing the REST API and ECHO-hosted browse images will submit images via ftp without metadata. Image deletions will be facilitated through the REST API. Collection or granule insertion will fail if a target browse record does not exist in ECHO. Browse image deletion will fail if one or more colections or granules reference the browse image. – Data partners not utilizing ECHO-hosted browse images will no longer manage browse images in ECHO as a separate process.

9. Sample Metadata Modifications Collection MODIS Aqua/Terra … ftp://lpdaac/:BR:1234.jpg … BR:123456 </AssociatedBrowseImages :SC:MOD10A1:12345 … ftp://lpdaac/:BR:1234.jpg … BR:123456 </AssociatedBrowseImages Granule MODIS Aqua/Terra … ftp://lpdaac/:BR:1234.jpg … http://api.echo.nasa.gov/LPDAAC/:BR:1234.jpg :SC:MOD10A1:12345 … ftp://lpdaac/:BR:1234.jpg … http://api.echo.nasa.gov/LPDAAC/:BR:1234.jpg References Browse Record Inline Browse Metadata Existing Proposed

10. System Impacts No Required Changes to ECHO Data Providers – ECHO data providers currently exporting ECHO 10 formatted metadata via ftp delivery with separate browse records will not need to make any changes to their exports. When the new Multi-Format Ingest capability is deployed, the existing FTP Ingest service will remain externally unchanged. Data Provider Transition from FTP to REST – It is proposed that the REST API not support separate browse records. Therefore, a provider transitioning their ECHO 10 exports from FTP delivery to the REST API will need to update their metadata exports accordingly. What You Sent Is Not What You Get (WYSINWYG) – A data provider exporting ECHO 10 formatted metadata may continue to utilize the separate browse records. However, the metadata that is ultimately ingested and retrieved by the new Multi-Format Ingest functionality will have collapsed the browse metadata into the Collection or Granule record. So, a provider ingesting via FTP will not be able to retrieve the same exact record via the REST API. Legacy Query Results – The legacy SOAP API query results will continue to return the same metadata. Translation to that format may be made simpler internally, but there will be no impact to existing ECHO. Simplified REST Query API – ECHO Clients that plan on utilizing the new REST Query API will only interact with collection and granule metadata records in each of the planned formats (ECHO, ISO 19115, Atom+Frost).

11. Questions?