1. Online data access in the SeaDataNet V1 system by Dick M.A. Schaap – technical coordinator Oostende, June 08

2. SeaDataNet Infrastructure versions and planning Version 0 = Continuation and maintenance of existing Sea-Search systems with minor modifications Version 1 = All data centres involved in upgraded metadatabases and transparent data access. Prototype ready by March 2008; all TTT partners by end 2008; thereafter gradually migrating all partners – operational by mid 2009. Version 2 = All data centres involved in Transnational Access and Access upgraded. Plus viewing services and further virtualisation. Operational by 2010.

3. Discovery services EDMED - Data Sets EDMERP - Research Projects CSR - Cruise Summary Reports EDIOS - Monitoring systems EDMO - Marine Organisations CDI - Common Data Index

4. SeaDataNet User’s portal schema V1 services for metadata Data downloading services Visualization services (WMS) for V2 Ifremer Database BODC Database BSH Database... Download managers in Data centers CSREDIOS EDMED CDI Shopping basket Requests status manager datametadata CSR CDI Project info Metadata & Data catalogues Data request Status of request General requestMetadata request Organisation + data source id Data download at BODCat BSHat MARIS + EDMERP Entry point for access hits EDMO EDIOSEDMED EDMERPEDMO Software Vocabularies StandardsCross search AAA Registr. My transact. User Register User registration

5. CDI is a fine-grained index to individual data measurements (such as a CTD cast or moored instrument record). This is in contrast to EDMED, which is an inventory of data sets, each of which comprises a large number of measurements. CDI paves the way for direct online data access through file downloads or online data requests. Present CDI V0 system: > 340.000 CDI entries from 36 Data Centres from 29 countries http://seadatanet.maris2.nl/v_cdi_v0/search.asp Common Data Index CDI

6. Common Data Index CDI V0

9. Central and homogeneous metadatabase, but for data access the users are confronted with the various user interfaces of the SeaDataNet data centres (which can provide online search and access / ordering facility / e-mail request) CDI V0 CDI url Data centre User interface User interface Data centre User interface Data centre User interface data

10. Central and homogeneous metadatabase plus transparent access to the distributed data sets via a unique user interface at the SeaDataNet portal CDI V1 CDI User interface Download Manager data Shopping basket Request Status Manager Download Manager Download Manager Central User Register

11. CDI Metadatabase and User Interface: Searching and browsing of metadata of data sets, managed by SeaDataNet Data Centres SeaDataNet Central User Register with User Roles db + web service : Details of users, their organizations and addresses, license period, Id-Passwords, SeaDataNet Roles of users SeaDataNet Shopping Basket: Preparing a user request of multiple data sets, handling validation of users and roles via the Central User Register, and routing requests to the Request Status Manager SeaDataNet Request Status Manager: Processing and administration of all requests and data deliveries (downloads), for users in communication with data centres. For data centres to oversee all transactions. CDI V1 System Modules

12. SeaDataNet Data centre Download Manager: Handles communication with SeaDataNet Request Status Manager and takes care that requested files are made ready for downloading by User (if cleared! ) in the agreed formats via a local website address. System Modules SeaDataNet Portal Data centre search shop Data download Metadata + transaction data

13. Order flow diagram Search in CDI directory Results Requests list Submit Requests (specify filetype? ) User Check CDI Meta database YES User + Role + Requests Shopping Basket NO Info about Conditions, Registration etc Log in Register Users + Licenses User Register User Roles WS

14. CDI V1 Search – home page

15. Search – Results page

16. Search – Detail page (full)

17. Search – records selected for Shopping basket

18. Shopping basket

19. Submit Shopping list => User Authentication

20. Authentication, Authorization and Administration Single Sign On system set up for access to distributed system User’s authentication information based on personal login / password User must register to get one login, thereby agreeing with the “SeaDataNet User Licence” Central User Register, but management is decentralized Authorisation based on “User Roles”

21. User identified and request for motivation and order confirmation

22. Request confirmed and link to RSM (also by e-mail)

23. Request Status Manager (RSM) Submit extended Request Form with motivation Shopping Basket YES User + Role + Requests Requests + user info in RSM; Split by DC Status: Pending Confirmation email to User with RSM info Validation of access via business matrix ABCD U V W Combi logic +

24. Data access restrictions in CDI V1 CCCollection cost charge FECommercial charge LILicence MOMoratorium NANo access NCDistribution cost OGOrganisation RSRestricted SRAcademic UKUnknown UNUnrestricted See Vocabulary L081 for definitions Multiple restrictions may apply

25. User Roles in AAA Register, of relevance for data access SDRN02Public SDRN03Academic SDRN04Commercial SDRN05National and local government SDRN06Pan-National government See Vocabulary C866 for definitions Multiple roles may apply

26. Business Matrix Restriction Role ABCD U V W OK TBD NO

27. Request Status Manager (RSM) Accepted requests in Download Manager Validation of access via business matrix Data Centre Status: OK Check in SDN User Register OK Retrieving files + preparing formats + giving URL for ZIP file Status: Ready for Download Status: TBD E-mail to DC manager Validate in RSM Status: OK Status: Access Denied Access Denied Not OK Download Manager Status: NO

28. Request Status Manager (RSM) Data Centre URL Downloading data sets directly from Data Centres Status: Downloaded Download Manager Retrieving files + preparing formats + giving URL for ZIP file

29. Request Status Manager and Download Manager RSM has Web service to communicate with the Download Managers, locally installed RSM keeps IP-addresses, EDMO codes, data manager e-mail, trigger addresses and V0/V1 labels of all connected Data Centres in config file Download Manager is Java programme, that must be locally installed Functions DM: Communication with RSM Web service Retrieving and processing order lists after being triggered Double checking user via AAA Web service Retrieving data files, related to requested CDI’s, via local_CDI_ID Preparing data files in agreed and requested formats (ODV, MedATLAS, NetCDF) Placing data files in zip format for downloading Informing RSM Web service of order processing status and url’s Keeping a local log First version DM = slim version => no NetCDF and use of pre-processed files Second version DM = full version => includes conversion of file formats and support for file management and database storage

30. Request Status Manager - Homepage

31. Submit Shopping list => User Authentication

32. User RSM – Overview of request per Data Centre and Status

33. User RSM : Status = Ready for Download + Download option

34. User RSM: Status = All

35. All submitted requests, their users, the data sets concerned, and processing of these requests is administered in the Request Status Manager SeaDataNet Data centres can also log on to the Request Status Manager and oversee all requests, handle ‘tbd’ requests and prepare a number of online reports. Monitoring & Tracking

36. Data centre RSM: overview Standing Orders

37. Data centre RSM: overview To be Discussed Orders

38. Data centre RSM: overview History Orders

39. SeaDataNet : URLs CDI V1 retrieval http://seadatanet.maris2.nl/v_cdi_v1/search.asp CDI V1 RSM http://seadatanet.maris2.nl/request/welcome.html --------------------------------------------------------------------------- CDI V1 RSM Web service TESTER: http://seadatanet.maris2.nl/test/rsmtest.aspx WSDL: http://seadatanet.maris2.nl/rsm/RequestStatusManager.asmx? wsdl

40. SeaDataNet V1 Implementation planning Prototype is ready It includes a temporary set of CDI records and data sets, that are managed by RNODC Majority of CDI test records is ‘unrestricted’, but some records have other ‘data access restrictions’ First version Download Manager installed => ODV ASCII files and use of pre- processed files See http://seadatanet.maris2.nl/v_cdi_v1/search.asp Use your SeaDataNet login - password

41. SeaDataNet V1 – Full implementation by TTT partners in 2008 Coming months all TTT partners will start installing the Download Manager and configuring their infrastructure for being a full V1 data centre. Preparing CDI V1 XML records following the upgraded CDI XML schema and using the new MIKADO software CDI must cover all available data sets Special attention to the ‘data access restriction’ Installing the Download Manager software Filling the coupling table between the LOCAL_CDI_ID (in the CDI XML records) and the local Data sets Choosing between pre-processed data files OR conversion via the Download Manager Note: data files can come from a local file management system OR from a local database via specific queries Must be ready end 2008

42. SeaDataNet interim V1 Implementation by other partners in 2008 Coming months all other partners will also start populating the CDI V1 database: Preparing CDI V1 XML records following the upgraded CDI XML schema and using the new MIKADO software CDI must cover all available data sets Special attention to the ‘data access restriction’ and LOCAL_CDI_ID Not yet installation of Download Manager, but data centres will be informed by the Request Status Manager by e-mail of all requests Data centres will process all requests manually using the Request Status Manager and indicating the URL where the user can download the data sets Must be ready end 2008; Effect is, that users will experience a common approach to all data centres. The difference is behind the scenes.

43. Request Status Manager (RSM) Validation of access via business matrix Data Centre Status: OK Retrieving files + giving URL for ZIP file Status: Ready for Download Status: TBD E-mail to DC manager Validate in RSM Status: OK Status: Access Denied Access Denied Status: NO interim V1

44. Request Status Manager (RSM) Data Centre URL Downloading data sets directly from Data Centres Status: Downloaded Retrieving files + giving URL for ZIP file Check by DC Manager interim V1

45. SeaDataNet full V1 Implementation by other partners in first half of 2009 Early 2009 all other partners will start with installation and configuration of the Download Manager, using the experience from the TTT partners: Installing the Download Manager software Filling the coupling table between the LOCAL_CDI_ID (in the CDI XML records) and the local Data sets Choosing between pre-processed data files OR conversion via the Download Manager Note: data files can come from a local file management system OR from a local database via specific queries Must be ready mid 2009

46. SeaDataNet V1 Demonstration See http://seadatanet.maris2.nl/v_cdi_v1/search.asp