1. Slide: 1 WGISS-37, April 2014 Yonsook Enloe Martin Yapur CWIC Report

2. What Is CWIC? The CEOS WGISS Integrated Catalog (CWIC) is being developed by WGISS to provide a framework that allows for easier search and access of Earth Observation data via partnering of CEOS agency data systems. For these partnering agencies, CWIC provides a single point of search and access. CWIC is a virtual clearinghouse of spatial, temporal, and science keyword metadata that brokers access to Earth Observation data by interoperating with data archives of other agencies and countries CWIC is envisioned to be a community catalog for satellite data that will implement the CEOS data systems harmonization results. 2

3. CWIC Information Architecture The CWIC architecture consists of 3 main items: CWIC middleware - provides an access point for an inventory-level search at CEOS agencies who are CWIC data providers CWIC clients/portals - user interfaces to access cross- discipline data from CWIC data providers IDN – International Directory Network (IDN) provides clients/portals directory-level search of CEOS agencies who have registered their collections in the IDN 3 IDNIDN CWIC Portal/Client

4. CWIC Architecture CWIC Mediator OpenSearch Service Interface Mediator Core I DN Legend CWIC – Wrapper CWIC – Mediator OGC CSW Protocol Java API Native Protocol OpenSearch API CWIC Clients OpenSearch /CSW Interfaces USGS Landsat Connector USGS Landsat Connector OpenSearch /CSW Interfaces NASA ECHO Connector NASA ECHO Connector OpenSearch /CSW Interfaces Brazil INPE Connector Brazil INPE Connector OpenSearch /CSW Interfaces CCMEO Connector CCMEO Connector USGS LSI Brazil INPE NASA ECHO Canada CCMEO Canada CCMEO OpenSearch /CSW Interfaces NOAA GHRSST Connector NOAA GHRSST Connector NOAA GHRSST CSW Service Interface

5. CWIC Basics Provides an access point for an inventory (granule) search at multiple CEOS agencies. CWIC provides translation from OGC CSW 2.0.2 (ISO) or “CEOS OpenSearch” to the agency inventory system protocol. CWIC uses “Connectors” to provide translation from CSW/OpenSearch to the agency system. CWIC does NOT contain any copies of data or metadata Search responses are returned using ISO 19115 or Dublin core metadata for CSW and Atom encoding for OpenSearch. Search responses contain urls to download data or urls to agency data ordering system or email orders. All CWIC data is searchable and accessible. CWIC clients expected to support directory/collection search at the IDN using CSW. 5

6. CWIC Team Who is on the CWIC team? CWIC Design team CWIC Server Implementation team CWIC Data Partners CWIC Client partners International Directory Network (IDN) Work together to solve common challenges for search and access – approaches and solutions that work for CEOS agencies’ operational EO systems. Need to consider end to end implications for assumptions and approaches, diversity of providers and clients, and scalability and maintainability of solutions

7. CWIC Support from USG At the October 2012 CEOS Plenary, Mike Freilich (NASA) announced that the US government agencies - NASA, NOAA, and USGS – will provide long term funding and resources to the CWIC activity starting in May 2013. The IDN existed before the CWIC initiative and provides other services to CEOS and GEO beside collection level discovery for CWIC. For this reason, at the present time the IDN funding will remain a contribution from NASA. 7

8. ISRO CWIC Data Partners CCMEO NASA AOE USGS INPE NOAA/ GHRSST NASA - National Aeronautics and Space Administration NOAA - National Oceanic and Atmospheric Administration CCMEO-Canadian Center for Mapping and Earth Observation GHRSST – Global High Resolution Sea Surface Temperature ISRO - Indian Space Research Organisation INPE – National Institute for Space Research USGS – United State Geological Survey AOE - Academy of Opto-Electronics, Chinese Academy of Sciences (CAS) -- (Beijing-1, NSMC/FengYun, CRESDA) 8

9. CWIC Data Partners & Data CWIC provides a major source of satellite data inventory search and data access in GEO. Current data partners include NASA, USGS (LSI), INPE, NODC/GHRSST, CCMEO, (dev partners: AOE, ISRO, NOAA CLASS, ROSCOSMOS,…EUMETSAT?) NOAA will add access to most of the data in CLASS when the new API is operational. NASA working to make all EO unrestricted online satellite data accessible from CWIC, including near real time data (from LANCE). GHRSST (SST VC) is working to make all their data accessible from CWIC at NOAA NODC. NODC will also add data to this CWIC connection. CCMEO provides search and email data ordering to access to Radarsat-1 & Radarsat-2 AOE working to add CRESDA (HJ1A, HJ1B, CBERS01, CBERS02, CBERS2B), Beijing-1 (BJ-1), and NSMC/Feng-Yun (FY3A, FY2D, FY2E). ISRO is making progress on their CWIC access (Nitant Dube). ROSCOSMOS to start work on their connections soon. EUMETSAT has scheduled an intro telecon in early May for data partner info

10. CWIC Data Partners API Consistency CWIC Data Partners API Consistency – making all the CWIC partners look “the same” – (makes for happier clients) for CSW interface and also for the “CEOS OpenSearch” interface Support same search criteria Support pagination Support chronological (most recent first) sort order for returned results Support online data access/order url or email order Support exceptions code handling (to convey to the user, the data partner specific info or other status) If spatial criteria is mandatory for provider, what the default spatial criteria should be (e.g. global). If temporal range is required, then send exception condition No “natural” default for temporal range. Details of each data partner API is on the CWIC webpage in the CWIC Connector API Report

11. CWIC Data Partner Responsibilities CWIC Data Partner responsibilities: Register all data collections (with CWIC accessible inventories) in the IDN Offer search access to the agency inventory system Offer data access to the granules/files to the CEOS/GEO users – o Direct data download o Direct link to data ordering system with specific data granule information o Email order with specific data granule information Provide technical POC (programmer type) who will assist the CWIC Server team in implementing a CWIC Connector that translates the CWIC supported protocol to their native protocol. Test the Connector and test access to collections of inventories Work with the broader CWIC team to come to agreement on resolutions to common issues. CWIC Data Partner do *NOT* need to Change native protocols Change metadata format

12. CWIC Client Partners CWIC-Start Prototype– user interface client for scientists (supports CSW) Provides collection search by IDN keywords (mission/instrument, science,..) via the IDN CSW Provides search and access to all CWIC accessible inventories via the CWIC Server CWIC-Smart Prototype – user interface client for scientists (supports OpenSearch) – accesses IDN & CWIC LSI Portal Provides search and access to the LSI data of interest COVE CCMEO Radarsat2 through the CWIC connection GEO Partners : includes GEO DAB, GENESI, GEO Portal

13. CWIC Support for New Partners Welcoming new partners! Bi-lateral introductory telecons – new partner can ask initial questions in “private” and not in full view of the BIG TEAM. CWIC Data Partners CWIC DocBuilder for CWIC DIFs – plus IDN staff assistance with questions. CWIC Server team builds “Connector” to translate CWIC searches to Data Partner system with assistance from new data partner. CWIC Server team assists with any testing issues, CWIC Client Partners CWIC Server team assists with any testing issues CWIC Servers : (DEV, TEST, PRODUCTION, STATIC) CWIC test clients

14. CWIC Road to OpenSearch Broad review of the OGC Geospatial and Temporal Extensions & OGC Extensions for EO documents in summer 2013 with ESIP OpenSearch implementers; Comments given to the CEOS OpenSearch team who submitted it to ESA (who funded the spec author). Most comments incorporated into the revised OGC spec released in late fall/winter 2013. Fall: Series of bi-weekly telecons with the CWIC, IDN, and CWIC- Smart teams, discussing an overall design for the OpenSearch implementation, identifying design/implementation issues, coming to resolution on these issues, etc. January 2014 CWIC team meeting, focused on making decisions on OpenSearch implementation issues and firming up the CWIC OpenSearch Best Practices.

15. CWIC Road to OpenSearch Feb – early April : OpenSearch implementation according to the CWIC OpenSearch Best Practices document at IDN, CWIC, and CWIC-Smart End March/early April – Early prototype implementations made accessible for team testing Future plans to compare the CWIC & IDN OpenSearch implementation interfaces with the FedEO OpenSearch interfaces; Major parts of the CWIC OpenSearch Best Practices will be put into the CEOS OpenSearch Best Practices wherever possible. Future plans to come to agreement within the CEOS OpenSearch team on the CEOS OpenSearch Best Practices and to revise the software, if necessary;

16. CWIC Status CWIC Servers (DEV, TEST, PRODUCTION, STATIC) working well on a new hardware platform at USGS/Sioux Falls since summer 2013. The CWIC servers at GMU will be shut down in early summer. Cwic.wgiss.ceos.org Cwictest.wgiss.ceos.org Cwicdev.wgiss.ceos.org Cwicstatic.wgiss.ceos.org Solidifying Test plans and processes to support new CWIC server enhancements, new data partner addition, IDN/CWIC synchronization, data accessibility testing (new data, confirmation of existing data); Automating testing wherever possible. Enhanced metrics collection and display

17. CWIC Status (cont) CWIC offers support for OpenSearch! CCMEO became an operational data partners (Radarsat-1 & Radarsat-2) The CWIC server team implemented email ordering support in the CCRS CWIC Connector – which populates an email with detailed information about the granule the user wants to order COVE (CEOS Visualization Environment) tool starting work to access Radarsat-2 metadata through CWIC CWIC-Start (new client prototype that implements OpenSearch)

18. CWIC Status (cont) IDN/CWIC synchronization plan simplified with the development of the IDN “tagging”/mapping tables. The metadata about metadata is kept separately as “tags” or mapping table. Example: Project=CWIC is now a CWIC tag. Late 2013, ROSCOSMOS announced plans to become a CWIC data partner April 2014, Eumetsat asked for more information about becoming a CWIC data partner using the OPenSearch API; Info telecon scheduled for early May

19. CWIC Future Work Work with the CEOS OpenSearch team to come to agreement on the CEOS OpenSearch Best Practices Align IDN/CWIC OpenSearch implementations with the final version of the CEOS OpenSearch Best Practices Interoperability testing with the FedEO OpenSearch implementation Study the feasibility of FedEO Connector to CWIC Produce new documents to update the CWIC Data Partner Guide, CWIC Client Partner Guide for OpenSearch. Support the developing CWIC Data Partners to become operational Strategy/plans for supporting VCs, WGD, SDMS, etc. for data access issues Continue to support GCI integration Strategy for “Service Level Agreements” – how to get agreements on improving the reliability of the CWIC service, including agreements with data partners

20. GCI Integration Both the IDN and CWIC integrated with GEO, and accessed by the GEO components (Geo Web Portal, DAB, GENESI). IDN provides collection metadata for over 24,700+ data collections available to GEOSS (April 2014), with 9600+ of these data collections tagged as “geossdatacore”. Registering a data collection in the IDN in effect registers the data collection in GEOSS. A data collection must be registered in the IDN to be CWIC accessible. CWIC provides access to over 1900+ (April 2014 figure) data collections with inventories (over 70 million granules, adding granules daily from live missions) Providing technical support for GEO component teams accessing IDN and CWIC. GEO Director, Barb Ryan, noted during the GEO session at the Nov 2013 CEOS Plenary, that of the 65 million granules accessible in GEO (Oct 2013), CWIC provided access to over 50 million granules.

21. GCI Picture

22. http://wgiss.ceos.org/cwic CWIC Project 22

23. CWIC Info and Resources http: wgiss.ceos.org/cwic Link to youtube CWIC demo How-to documents : CWIC Data Provider, CWIC Client Provider, CWIC DIF Guide CWIC Connector API Report, CWIC Exceptions Handling Guide,… Link to CWIC DIF Builder tool Links to CWIC enabled clients Links to open source CSW software (for Partners) Link to Test Environment cwic@wgiss.ceos.orgcwic@wgiss.ceos.org email list Monthly CWIC Team telecons

24. Farewell to Dr. Yuanzheng Shao Yuanzheng Shao, the development lead for the CWIC Server is leaving the team to return to China. Shao will be an associate professor at the University of Wuhan. THANKS very much for your insights and hard work! Our very best wishes to you and to your family in your new life in China!