1. Distributed Marine Data System:
Intercommission Coordination Group on WIS
IODE/JCOMM
Distributed Marine Data System:
E2EDM and Ocean Data Portal Pilot Projects
Nikolay Mikhailov,
IODE/JCOMM ETDMP Chair,
RIHMI-WDC Russian Federation
Brazil, July 2008

2. The IODE/JCOMM initiatives
Data&information exchange is important for World Ocean studies – on regional and global levels
Observation systems
The IOC IODE system forms a worldwide service-oriented network consisting of NODC’s, DNA’s, WDC’s.
The IODE/JCOMM initiatives
for Distributed Marine Data System development
IODE/JCOMM E2EDM technology prototype Pilot Project ( )
IODE Ocean Data Portal (2007-…)
Data and product delivering systems
Delay-mode data base’s
Analysis and Forecasting systems

3. IODE/JCOMM Ocean Data Portal/E2EDM Technology
Vision
“The Oceans Data Portal will deliver a standards based infrastructure that provides integration of marine data and information from a network of distributed IODE NODCs as well as resources from other participating systems.“
Ocean Data Portal supports the data access requirements of all IOC programmes areas, including GOOS, JCOMM, HAB and the Tsunami warning system.
Ocean Data Portal will provide the JCOMM WIGOS operations

4. Data and Services
The Ocean Data Portal will provide on-line access to the marine data and information resources of the participating data centres including:
operational and delayed-mode data
data and services from the oceanographic and marine meteorological domains
from multiple data source formats and local data systems (DBMS, data files, map services, electronic documents)
from multiple data providers in different geographic regions

5. ODP/E2E Distributed Marine Data System Principles
The ODP will be interoperability with existing systems
IODE data centres will need to accept and implement a set of agreed interoperable arrangements including the technical specifications and Web-services
Interoperability will be achieved through the use of internationally endorsed standards (such as SOA, ISO and OGC)
NODCs will not be required to change their internal (local) data management systems

6. ODP/E2E Technical Concept
IODE Portal
Portal of system 1
(e.g. SeaDataNet)
Portal of system N
(e.g. WIS)
Network of distributed IODE data centres
Portal Web-services
Metadata
Data
Internet
Dedicated data sources network- E2E

7. E2EDM Architecture Software Metadata Use External layer E2EUserMD
E2EInterfaceMD
Discovery
- E2ESeachMD,
Access
- E2ELinkMD
Integration Server layer
Encoding
E2EElementsMD,
- E2ECodesMD
Request/Responce
- E2E Request,
- E2EResponce,
- Transport Data File (Point, Profile, Grid) – NetCDF, ASCII, XML
Communication layer
Config.file
Mapping file
Data Provider layer
Local data system layer

8. Model of IODE/JCOMM Distributed Marine Data System
E2E Integration Server
Monitoring, Administrating, Discovery, Interaction services
Users
Ocean Data Portal
Publishing services (end-user applications)
E2E Data Provider
DBMS, Data files (structured), Object files (documents,…)
Distributed Marine Data System
IODE/JCOMM Data Centres – national and specialized
Local Data Systems
System Coordinator Centre – global, regional, task-oriented
Data Access

9. Model of IODE/JCOMM Distributed Marine Data System
Portal
Integration Server
Integration Server
Integration Server

10. IODE/JCOMM Distributed Data System operation (current status)
User access
Portal
HTTP communication
API‘s
Registry
Metadata management Data discovery
Data Delivery
Monitoring & Security
E2E Integration Server
Registry
Connection
Mapping
Data Access
HTTP communication
Web-services
Data providers community
E2E
Data Provider
Request message
Response message
netCDF transport
data-file
Local Data system
Data Centers Network

11. IODE/JCOMM Distributed Marine Data System coverage
Data Provider
(VLIZ)
Data Providers
JCOMM PAs
RIHMI-WDC mirror
Integration server
Integration server
operational
planned
finished PP
IODE Project Office
Data Provider
(RIHMI-WDC)
Data Providers
IODE ODINs
Data Provider
(IFREMER)
Data Provider
(MetOffice)

12. E2E Integration Server operation

13. E2E Data Provider operation

14. IODE Ocean Data Portal prototype

15. IODE Ocean Data Portal prototype
Providing some Google-like search

16. IODE Ocean Data Portal prototype
- Result of data search

17. IODE Ocean Data Portal prototype
- Advanced metadata search

18. IODE Ocean Data Portal prototype (WOD, North sea)
User data view page

19. IODE Ocean Data Portal prototype
User data view page – different type of products

20. E2EDM Technology Documentation
Title
Contents
Version and updating
E2EDM Overview
General information about E2EDM technology and implementation of this technology
Jan 2007
Updated May 2008
The Checklist of technical requirements for data providers for E2E technology implementation
Describes requirements for IODE data centres to build the E2E Data providers network
E2EDM technical specification v.1.0
Describes technical specifications (model, metadata structures, data exchange protocol and etc.) the E2EDM technology
(TO BE UPDATED!!!)
Updated 15 Jan 2007
E2EDM Concept XML Schema
XML document includes ocean and marine meteo parameters which circulates under the distributed data system (current data source federation).
Updated 16 November 2007
E2EDM Global XML Schema
Includes XML structures of metadata classes (packages) used for E2EDM metadata records construction.
Updated 14 October 2007
E2EDM User's Manual
Document is considered as quick-start guide for the E2EDM technology users (general architecture of the E2EDM technology, installation and use of the E2EDM Data Provider and Integration Server software). The document is intended for data centres to built the distributed data system.
E2EDM Integration Server User Guide
Includes the detail rules and procedures how to install, configure and run E2EDM Integration Server software.
v.1.0., updated 05 May 2007
E2EDM Data Provider User Guide - Installation and setup
Describes detail rules, procedures and web-forms for the Data Provider installation, registration and maintenance. The document is intended for system administrators of the data centres and other institutions responsible for providing data to the system.
v September Updated May 2008
E2EDM Data Provider User Guide - Registration and maintenance of resources.
Describes detail rules, procedures and web-forms for registration and maintenance of the local resources of distributed data sources system. The document is intended for operators of the data centres.
E2EDM Ocean Data Portal. User Guide
Describes detail rules, procedures and web-forms for use Ocean Data Portal basing on the E2EDM technology. The document is intended for end-user for enter and access to resources of the distributed data system.
v1.0 Updated December 2007

21. Links to ODP/E2E
E2EDM Web site – documentation
E2EDM Integration Server
- IODE Project Office (VLIZ)
- RIHMI-WDC (mirror)
JCOMM/IODE Data Portal -
JCOMM/IODE Data Portal web-site will be launched soon

22. Interaction with other systems (WIS case)
OAI Client
(GISC)
OAI Server
DCPC
Obninsk
WMO
Schema
«Register»
WMO metadata records
(all with global unique ID)
«Request»
«Response»
E2EDM records are reformatted into WMO standard
records are migrated from (A) to (B) A B
GISC catalogue / registry
Resource catalogue
E2EDM Integration Server)
E2EDM Data Provider
VGISC protocol communication

23. The IODE/JCOMM DDS organization
The dedicated data center should support the Integration Server operations
The data center which agrees to be the E2E data provider should provide:
the appropriate middleware for communications: web server and application server
installation of the E2E Data Provider software to connect the local data system with E2E Integration Server
registration of the data source and its resources
support of the local data system. .

24. (IODE Officers and DMCG-3 meetings)
ODP Future Plan
(IODE Officers and DMCG-3 meetings)
Establish the pilot project working group – in process
Draft of a detailed work plan – Oct.2008
The JCOMM WIGOS basing on E2E Technology with expanding the DPs –
The Design Docs and take into account other similar initiatives to avoid duplication of effort – beginning 2009

25. Thank you !