IODE Ocean Data Portal updates Nikolai Mikhailov, Sergey Belov, Natalya Puzova 6th Session of the IODE Steering Group for the ODINBLACKSEA project 6 - 8 November 2017, Oostende, Belgium

Objectives International Oceanographic Data and Information Exchange (IODE) IODE facilitates the exchange of oceanographic data and information between participating Member States, and serves the needs of users for data and information products. IODE Ocean Data Portal (ODP) Facilitate and promote the exchange and dissemination of marine data and services; Provide the seamless access to marine data to NODCs across the IODE network through the discovery, evaluation and access to data via web based services; Identify and recommend standards to provide interoperability with IODE data centres to allow shared use of metadata, data and products. As many of you may know, The International Oceanographic Data and Information Exchange (IODE) was established by in 1961. It strives to enhance marine research, exploitation, and development through exchange of data and information between member states. Supports a variety of programs including standards development, technology, data access, capacity building (education). Adheres to the IOC data policy – Free and open access to data and information (click) IODE Ocean Data Portal is one of the IODE projects, formally established in 2007 and started its operation in 2009. The major underlying principles of the IODE Ocean Data Portal are Interoperability, Non-intervention and Free access to data and services. In 2013, a resolution was passed that will result in all member states of the IODE to enable access to data through the ODP, so the ODP is now considered as an IODE platform for data share and exchange. 2

Benefits of the IODE ODP The ODP provides benefits to both data providers and data users: Benefits for data providers Scalable environment to support the capacity of the data provider’s environment Supported technology for data discovery, evaluation, and access Standards (discovery metadata, vocabularies, code lists) Improved interoperability with other major marine data systems Access to advice and support from a team engaged with relevant technical and data management expertise within the global community Benefits for data users One-stop shop for data and web services (discover, access, download) Increased visibility for one’s research data Increased visibility of the researcher within the marine domain (Ocean Experts) The role of the ODP is to simplify the process of making data openly available to the marine community through the delivery of supported technical solutions (software), support for data providers, and training. Following benefits can be considered from the data providers prospective while sharing data and information through the IODE ODP: Scalable environment to support the capacity of the data provider’s environment It Promotes defacto standards Is Engaged with other IODE data systems Supported technology for data discovery, evaluation, and access Improved visibility of data and geospatial web services Improved interoperability with other major marine data systems Operational assistance and support from the technical point of view (click) The benefits for data and information users (I am broadening those also meaning products and services) includes: One-stop shop for data and web services Increased visibility for one’s research data Increased visibility of the researcher within the marine domain (Ocean Experts) But one of the major benefits for both sides, even sometimes invisible for them, is that The ODP adopts and support major standards from ISO and the Open Geospatial Consortium (OGC) in order to maximize the interoperability between the ODP and other major data systems. By doing this, ODP can provide access to data from many major platforms within a single portal. 3 Standards

INTEROPERABILITY PACKAGE TECHNICAL AND TRAINING SUPPORT ODP Components ODP TOOLKIT ODP PORTAL INTEROPERABILITY PACKAGE TECHNICAL AND TRAINING SUPPORT DATA AND INFORMATION The inhomogenity of data sources, data storage types and formats and it's integration in most cases will be separated on individual issues to deal with. Not in case of IODE ODP. ODP is delivering tools to connect and describe such data and information, provide capabilities to discover it, present and download. So called ODP toolkit is developed to handle these functions and far more beyond them. One the major components within OPD is an interoperability package called to provide interoperability arrangements, as well as means and tools for system to system conversation. Such components are becoming very critical to reduce or even avoid duplication of efforts when required data and information already exists and provided by some programme, project or initiative. 4 4 4

ODP Components The ODP is comprised of 3 major classes: ODP Toolkit Interoperability package Technical and Educational Support Metadata “crosswalk” rules Discovery metadata services Machine-to-machine Interfaces Partnership Centre for ODP – support and maintenance IODE Ocean Teacher courses (online materials, video courses, etc.) Integration Discovery Access Download Dissemination Control Standardization Scalability …… The ODP is comprised of 3 major components: ODP toolkit – plug-and-pay base back-office software stack supports the deployment of ODP nodes via virtual machine packages maintained by the ODP Partnership Centre. It also includes Portal - web portal component that may be extended and stylized to support node requirements. Interoperability package – specialized interoperability arrangements developed between ODP and other major systems such as SDN, WMO, EMODNet Technical and Training Support - In parallel with the physical infrastructure components, the Partnership Centre for the ODP provides development/technical support and training support through the IODE Ocean Teacher Academy.

ODP Toolkit Open-source based components: Java JBoss PostgreSQL + PostGIS Geoserver, Grass GIS, Python, Perl OpenLayers, Leaflet Geonetwork Jboss Portal Platform JOSSO Zabbix Security Management Monitoring Service Bus Portal Presentation Portlets GIS server Data Cache Integration, delivery, exchange Integration Server Data Provider

ODP Toolkit Standards used: ISO 19115/19139 NetCDF OGC WMS, WFS OGC CSW RESTful services SOAP web-services Security Management Monitoring Service Bus Portal Presentation Portlets GIS server Data Cache Integration, delivery, exchange Integration Server Data Provider

ODP network of data providers ODP Toolkit – Service model queries consumers results Discovery interfaces orders ODP network of data providers FTP Delivery Services Email HTTP push Data services SOAP HTTP REST CSW FTP pull OGC WxS

Use Case: SNDM Argentina Sistema Nactional de Datos del Mar (SNDM) www.datosdelmar.mincyt.gob.ar SNDM is an initiative of the Ministry of Science, Technology and Innovation in conjunction with the Interagency Council on Science and Technology ( CICyT ) framed within the Program Large Instruments and Databases SNDM is responsible for ensuring accessibility to data and marine information and Southwestern Atlantic Ocean Antarctica.

Latest technical developments. New portal and search capabilities As the result of the delivery of updated ODP Portal, data search and access has been changed from ad-hoc query base to interface with DataCache. This implies that when updated data sets and their corresponding metadata from Data Provider and Integration Server are loaded to DataCache database (automatically on event) they can be used afterwards for data viewing, download and processing (spatial analysis, merging, etc.).

Latest technical developments. New portal and search capabilities application for viewing and searching metadata at ODP catalogue stored in DataCache, managing subscription for data delivery to email and/or FTP on event base and for registered and authorized users application for accessing data from datasets and displaying it in map, chart and table forms, downloading complete datasets or subsets application for web GIS viewer for services published in ODP GIS server and from external geo services (OGC WMS, OGC WFS, ArcGIS, geoRSS, etc.)

Latest technical developments. GIS and DataCache ODP GIS services On-fly OGC WMS/WFS/WCS creation Web GIS ODP DataCache DataCache is providing following functionality: Event-based metadata catalogue management (add, update, delete); Event-based data loading to database; Spatial operations using PostGIS and plperl; Operations with loaded data using workflow schemes (calculations, analysis, merging and splitting, etc.); Data store for portal services.

Latest technical developments. API and services SOAP web services to search and retrieve metadata and data REST web services to search and retrieve metadata and data JSON services to search and retrieve metadata and data Examples: http://is.oceandataportal.net/iserv2/metadataJSON?sSearch=SYNOP http://is.oceandataportal.net/iserv2/dataJSON?action=getData&id=RU_RNODC_63

Use Case: SNDM Argentina ODP technology is used to build national marine data system for Argentina - Sistema Nactional de Datos del Mar (SNDM) http://portal.mincyt.gob.ar The system was officially launched in 2013 and the ODP team continues to work with the client team in order to improve SNDM metadata model and other facilities. Importance of this use case: Example of full implementation of regional node and virtual data providers. Contributed to the advancement of ODP packaging to simplify implementation and support. Demonstrates flexibility of ODP technology to support other themes. Contributed to the development of the ODP training offerings. In addition to the fact that ODP has been tested and widely used for different national projects in Russia, where it was developed and still supported by the Partnership Centre for IODE ODP, one of the real examples of using ODP as integration platform is SNDM of Argentina which connects 9 institutions around the country, holding significant data and information holdings to be submitted into the national data system, and later to be harnessed with other disciplinary national systems (biology, magnetics, etc.) Importance for this use case consists in: Example of full implementation of regional node and virtual data providers. Contributed to the advancement of ODP packaging to simplify implementation and support. Demonstrates flexibility of ODP technology to support other themes. Contributed to the development of the ODP training offerings. 14

IODE ODP project progress Following nodes have been established IODE ODP Global node (RIHMI-WDC) IODE ODINWESTPAC Regional node (NMDIS, China) IODE ODINAFRICA Regional node (KMFRI, Kenya) IODE Regional/national node of Sistema Nacional Datos del Mar (SNDM, Argentina) 16 data providers from Russia, Argentina, Kenya, Canada, USA and other countries providing access to more than 500 data sets. IODE ODP contributes EC Horizon2020 Ocean Data Interoperability Platform project by providing access to harmonized data and information from EU, USA and Australia (http://www.oceandataportal.net/portal/portal/odp2/interoperability). In 2016 Russian segment of GOOS has been created based on IODE ODP http://www.oceandataportal.net/portal/portal/goos/data

IODE ODP project progress Implementation of interoperability with SeaDataNet as a deliverable of the Ocean Data Interoperability Platform (ODIP) Ongoing collaboration and access to data and services from WMO WIS, EMODNet Delivery of training sessions to groups including ODINWESTPAC, ODINAFRICA (5 training courses) Enhancement of the ODP technical environment in order to improve reliability and support (ODP V2 toolkit) Ongoing publication of new data sets

Options for ODINBlackSea Deployment of regional node Virtualization of regional node ODP regional node in Turkey (NODC) Local infrastructure and staff support Training support by ODP PC and OTGA Deployment support by ODP PC Node evaluation and launch support by ODP PC National nodes Physical installations in centres (using ODP V2 toolkit) or virtual data providers on regional node ODP regional node in ODP global node (RIHMI-WDC) no support for relational databases data should be provided in files via HTTP, FTP, upload Training support using IODE OTGA National nodes Physical installations in centres (using ODP V2 toolkit) or virtual data providers on global node

Proposed action plan Extensive list of planned data types and data sets (by all participants) Installation plan for data providers (by all participants) Installation plan for ODINBlackSea regional node (by coordinator) Training arrangements (by ODP PC) Training courses (1Q 2018, ODP PC)

References International Oceanographic Data and Information Exchange (IODE): http://www.iode.org Ocean Data Portal Web Site: http://www.oceandataportal.org Ocean Data Portal Global Portal: http://www.oceandataportal.net SNDM-Argentina Regional ODP Node: http://portal.mincyt.gob.ar/portal/ ODINWESTPAC Regional ODP Node: http://portal-odp.nmdis.gov.cn You are very much welcome to visit some links presented on this slide. 19