NATO ARW September 2007 SeaDataNet A Pan-European Infrastructure for Ocean and Marine Data Management by Dick M.A. Schaap (MARIS) Technical coordinator

NATO ARW September 2007 Background Data availability of vital importance for marine research European data set is fragmented Not always quality controlled Not always easily accessible In 35 countries bordering European seas more than scientific laboratories from government organizations and private industry collect data use various sensors on board of research vessels, submarines, fixed and drifting platforms, airplanes and satellites measure physical, geophysical, geological, biological and chemical parameters, biological species, etc.

NATO ARW September 2007 What is SeaDataNet? EU Sixth Framework Programme Integrated Infrastructure Initiative (I3) 1 April 2006 to 31 March 2011 Objectives develop an efficient distributed pan-European marine data management Infrastructure for managing large and diverse data sets network existing professional data centres of 35 countries, active in data collection provide integrated databases of standardised quality on-line

NATO ARW September 2007 SeaDataNet partners 49 partners including: Major oceanographic institutes of 35 countries National Oceanographic Data Centres (NODCs) Divisions of major national marine research institutes 3 satellite data centres 3 scientific modelling centres 2 SMEs experts in software development 3 International organisations (IOC, ICES, JRC)

NATO ARW September 2007 Building on experience Leaders of Data Management Concerted Actions MEDAR/MEDATLAS II - Mediterranean Data Archaeology and Rescue, aiming to make available a comprehensive data product of temperature, salinity and bio-chemical data in the Mediterranean and Black Sea through a wide international cooperation, toSea-Search – The gateway to Oceanographic and Marine Data & Information in Europe, to provide an effective navigation tool to oceanographic and marine data and information sources for European marine waters and to centres in Europe with expertise and skills in oceanographic and marine data & information management. Data management of scientific projects: MTPII-MATER, CANIGO, OMEX, MERSEA, etc. Bringing in new partners – satellite data centres

NATO ARW September 2007 SeaDataNet components

NATO ARW September 2007 Infrastructure, versions & planning Version 0 Continuation, maintenance and further population of existing Sea-Search systems with minor modifications Version 1 All data centres involved in upgraded meta-databases (ISO compliant XML metadata-profiles and web services) Technical Task Team partners involved in transparent data access, security, and monitoring of use = trans-national access via downloading of datasets in standard formats (ODV ASCII; NetCDF; MedAtlas) Launch by March 2008 Version 2 All data centres involved in trans-national access and upgrading of data access to include spatial data infrastructure features (adopting OGC; ISO and INSPIRE standards) and using XML data profiles Operational by 2010

NATO ARW September 2007 SeaDataNet Data Policy Metadata free and open access, no registration required each data centre should provide the meta-data in standardized format to populate the discovery services (metadata directories) Data and products services in general free and open access, but mandatory registration a “SeaDataNet role” (partner, academic, commercial etc.) is attributed to each user Each NODC attributes the roles to the users of its of country Out of the partnership, the roles are assigned by SeaDataNet user- desk each data centre node delivers data according to the user’s role and its local regulation each data centre should provide freely the data sets necessary to develop the common products

NATO ARW September 2007 Discovery services Marine environmental data sets (EDMED) Marine environmental research projects (EDMERP) Observing systems (EDIOS) Research Cruise Summary Reports (CSR) Common Data Index (CDI) Directory of marine organisations (EDMO)

NATO ARW September 2007 Cruise Summary Reports Conceived by IOC ICES acted as focal point Currently over than cruises Documenting data collected on scientific research cruises operated by European vessels Cruise track charts included Online entry form Online searchable database

NATO ARW September 2007 European Directory of the Ocean- observing System (EDIOS) Initiative of the European Global Ocean Observing System (EuroGOOS) Regular repeated measurements On-line searchable directory of observing, measuring, and monitoring systems Over 10,000 observation entries Links to on-line real-time and archive data Tool to aid design of observing systems

NATO ARW September 2007 European Directory of Marine Organisations (EDMO) Over organisations: addresses, profiles and relations in other directories Online maintenance per country by CMS On-line user interface with combined alphanumeric – geographical searching Export in.csv files Also available as Web service (SOAP based)

NATO ARW September 2007 Common Data Index Index to individual data sets Now > entries from 20 partners Mid Oct 2007 entries from 35 countries ISO19115 metadata standard has been adopted On-line searchable directory Direct access to on-line data sets, on-line order facility or request

NATO ARW September 2007 Technical developments Streamlining of metadata formats for interrelationships between the directories Establishment and governance of common vocabularies ISO content model as basis for the XML formats and exchange schemas (XSD) with SeaDataNet XML approach Innovating maintenance modalities: XML exchange using local software component (MIKADO) or own application XML validation services Online Content Management System (CMS) Introducing Web services Vocabularies and Directory of Marine Organisations (EDMO) Coming: other metadata directories

NATO ARW September 2007 Interrelationships between the directories, using vocabularies Country Param Platf … … … EDMED EDIOS EDMERP CDI CSR EDMO XML

NATO ARW September 2007 Common Vocabularies Controlled vocabularies containing keys, terms, abbreviations and essential (but often missing) definitions Lists for platforms, countries, sea areas, etc… International content governance by SeaVox list server Web service On-line user client Export in.csv files

NATO ARW September 2007 SeaDataNet standards Interoperability – key to distributed data management system success Common understanding of terms used via common vocabularies Standard data exchange formats (ODV ASCII, NetCDF, MedAtlas) Common quality control protocols and flag scale Standard Web services Schemes for handling duplicate data Ocean Data View software (ODV) adapted for SeaDataNet

NATO ARW September 2007 SeaDataNet : Delivery Services V1 Primary function Users will be able to download selected data sets CDI User Interface Searching and browsing of metadata of data sets

NATO ARW September 2007 SeaDataNet : Delivery Services V1 Shopping basket Preparing a user request of multiple data sets Central User Register Validation of users and their roles Business matrix Combining user role and data access restriction(s) to decide on modality for processing each data request ABCD U V W

NATO ARW September 2007 Central User Register Single Sign On system required for access to distributed system User’s authentication information based on personal login / password User must register in order to get login Web form to provide necessary information Online user agreement “SeaDataNet General Licence” Registration processed per country via online CMS After processing, login/password sent by ( check) User gets ‘Role’ SeaDataNet Data Policy Data access based upon combination of User role - Access restriction(s) as indicated in CDI record and business logic

NATO ARW September 2007 SeaDataNet : Delivery Services V1 Request Status Manager Processing of all requests and data deliveries Enables users to follow progress Enables DC’s to oversee transactions Download manager Local component for communication with portal and preparing data deliveries

NATO ARW September 2007 SeaDataNet User’s portal schema V1 IFREMER Database BODC Database BSH Database... Download managers in data centres CSREDIOS EDMED CDI Shopping basket Requests status manager datametadata CSR CDI Project info Metadata & Data catalogues Data request Status of request General request Metadata request Organisation + data source id Data download at BODCat BSHat MARIS + EDMERP services for metadata Data downloading services Visualisation services (WMS) for V2 Entry point for access hits EDMO EDIOSEDMED EDMERPEDMO Software Vocabularies StandardsCross search AAA Registr. My transact. User Register User registration

NATO ARW September 2007 Data products SeaDataNet will provided added-value products to be created by regional centres Mediterranean Sea Black Sea Baltic Sea Arctic and North Sea Atlantic and global ocean First products => gridded climatologies based on already available data-sets (mostly hydrographic data) updated during the project life To make products comparable, all regional data centers will use a common tool for spatial mapping For this a new version of DIVA (Data Inverse Variational Analysis) was developed (University of Liege)

NATO ARW September 2007 SeaDataNet wider involvement SeaDataNet partners are involved in many other European and international projects, contributing and safeguarding good data management. Examples: Services for the European marine geological institutes EDIOS Services for EuroGOOS Partners in the MERSEA project (GMES) for MIM Partners in the HUMBOLDT project (GMES) for developing a framework for a European Spatial Data Infrastructure Partners in SIMORC for oil & gas industry metocean data Partners in ECOOP for data provision and management Component of the GMES Marine Core Services

NATO ARW September 2007 Marine geology data Catalogue of > seafloor samples and >2.5 million line kilometres of marine seismic and sonar survey from the ocean basins and continental shelves, held at European institutions

NATO ARW September 2007 Industry metocean datasets - SIMORC System of Industry Metocean data for the Offshore and Research Communities Metadatabase and Database of datasets from major oil & gas companies At present ca 870 data sets (Shell, BP, Total), more underway

NATO ARW September 2007 SeaDataNet - BlackSeaScene BSS is an Integrating Activity of EU-Research Infrastructures programme and runs from 1 Dec 2005 – 30 Nov 2008 BSS has 32 partners with 25 from Black Sea region and is coordinated by MARIS BSS is a Black Sea Scientific Network analysing the present quality of Black Sea datasets and applied DQC procedures developing a Black Sea regional portal for metadata and data access BSS is adopting the SeaDataNet standards and building upon the SeaDataNet basic services BSS engages 25 Black Sea institutes, strengthening and expanding the compilation of metadata and access to data for the Black Sea region

NATO ARW September 2007 SeaDataNet capacity building The training program has two main objectives: to ensure that the data and metadata to be integrated in the SeaDataNet system will be formatted, checked for quality and disseminated according to the common protocols developed to transfer expertise and to train IT experts of the data centers in the basics, installation and operation of the SeaDataNet technical components

NATO ARW September 2007 International Conference IMDIS 2008 Second international conference “International Marine Data Information Systems” (IMDIS) March 31 - April 2, 2008, Athens, Greece Joint organisers: SeaDataNet, Hellenic Centre for Marine Research, IOC/IODE Four sessions: Marine environmental data bases: infrastructures and data access systems Data Services in ocean science: standards and interoperability Visualisation tools Services for Users and Education

NATO ARW September