ODP V2 Data Provider overview

22 Scope Data Provider provides access to data and metadata of the local data systems. Data Provider is a wrapper, installed in the local data system and becomes a data source for the distributed data system. The Data Provider processes the local data sets and in a semi-automated way generates the discovery metadata.

33 How to become a data provider The data center which agrees to be the ODP data provider should provide:  the middleware for communications: application server available through HTTP protocol,  installation of the ODP Data Provider software OR use light Data Provider extension,  registration of the data source and its discovery metadata,  support of the local data system

4 How to become a data provider Light Data Provider extension: Allows integration of data from data centres unable to install the Data Provider software. In this case the owner of the Data Provider must create new user with login and password and provide this information to remote user Data centres can use remote Data Provider for providing catalogs of data to ODP distributed system

55 Supported data storage types Data in relational Database Management System (DBMS): Oracle, MS SQL Server, MySQL, PostgreSQL, etc.; Structured data files with non-hierarchical data formats (CSV, TSV, …) Object data files: documents, images, data which has format not supported by ODP technology Links (web site pages, web-applications, URLs, web- services)

66 How to provide the data Data in database inside local network with the Data Provider software Structured data files: upload to the Data Provider server or specify URL to data files location (FTP, HTTP) Data Provider DBMS files Local network of Data Centre FTP, HTTP Internet

7 Network requirements HTTP and SOAP protocols must be available JBoss AS port must be opened in firewall settings IP-address verification: Data Provider receives requests only from the Integration Server

8 Software requirements Middleware software J2SDK 7 Application server JBoss GA

9 Data exchange protocol Request and response messages: XML document via HTTP or SOAP protocol Transport data file : NetCDF file for data from DBMS and structured files, data file “as is” for object data files.

10 Functional requirements The local data administrator should provide: design of resources; data source registration; discovery metadata registration; provision of data

11 How to prepare metadata (design of resource) This work can be handled before or at once after the Data Provider installation and includes the following actions: local data structure analysis assessment of resource contents assesment of data granularity discovery metadata content (title, area coverage, temporal extent, …)

12 Local data structure analysis Local data storage types: SQL-oriented Database Management System: Plain table structure – 1 parameter in 1 column. P1P2P3... Pn P1 P2 P3 … Joining more than couple of tables is not a clever solution – use views if it’s possible

13 Local data structure analysis Plain structured data files (CSV, TSV, etc.). List of formats is extendable by adding new data parser; Structured data file can include comment lines, different data value separator; Objective files – no restrictions. Object files must be accessible via HTTP or uploaded to the Data Provider server. //M4321;M4322;M4401;M4400;M4050;M4311;M4312;M4303;P0696_00;P0735_00;P0964_00;///P0507_00;P 0001_00; //Forecast from 00 hr Producing center - NCEP/NOAA ;999.; :00; :00;0; ;0.0000;0;0.6;-9.1;0.01;-37.3;-36.6; ;999.; :00; :00;0; ;0.3125;0;0.7;-9.1;0.01;-37.3;-36.6; ;999.; :00; :00;0; ;0.6250;0;0.8;-9.1;0.01;-37.2;-36.6; ;999.; :00; :00;0; ;0.9375;0;0.8;-9.1;0.01;-37.2;-36.6; ;999.; :00; :00;0; ;1.2500;0;0.9;-9.1;0.01;-37.2;-36.6;

14 Resource granularity The Data Provider has the possibility to support various types of the data granularity, i.e. it is possible to create discovery metadata, search and deliver full datasets or specified data pieces (logical data units) of datasets - single cruise or data profile, single buoy or single coastal station data - and etc. The resource can be presented as a single unit (called a single resource) or as a set of resource instances (called a serial resource) reflecting the local data granularity of the local data system. The data granularity level is adjusted in the process of the local dataset registration.

15 Resource life-cycle define the schedule for updating of discovery metadata, check the local data availability using the report submitted by Integration Server, take the needed actions to provide the data source actuality (connection, data files storage availability),

16 How to generate metadata Use web-interface of the Data Provider to generate, update and maintenance discovery metadata

17 Thank you for attention! 17