1 Grid DAIS: Database Access and Integration Services Greg Riccardi Florida State University

2 Overview of Presentation Goals of DAIS Conceptual model of Grid database access Examples of client-service interactions  Discovery and creation of services  Asynchronous query processing and datasets  Updating from datasets Representing Sky Query in DAIS Other topics/issues

3 Goals of DAIS The group seeks to promote standards for the development of grid database services, focusing principally on providing consistent access to existing, autonomously managed databases.  Provide service-based access to existing data management systems.  Accommodate several widely used data management paradigms (e.g., relational, object, XML) within a consistent framework.  Provide sufficient information about itself to allow the service to be used given the specification of the service and the metadata provided by the service.  Peacefully coexist with other Web and Grid Service standards.  Be orthogonal to Grid authentication and authorization mechanisms.  Support higher-level information-integration and federation services.

4 Desirable Properties of DAIS Systems OGSI/A compliant  Letter and Spirit Plugability/Extensibility  Different kinds of data resources  Many access mechanisms  Evolvable Easy to understand and apply  Existing standards/designs  Tooling GridServices and WebServices applicable Supports current technology Access AND integration  Integration of different models at the data level Implementable Integrateable into customer scenarios Technology independent

5 The Model – External Artifacts DBMS DB Resultset External data resource manager External data resource External data set External = external to the OGSI compliant grid

6 The Model – Logical Artifacts DBMS DB Resultset data resource manager data resource data activity session data request data set

7 Data Resource Manager External data resource manager (edrm)  A data management system such as a relational database management system or a file system Data resource manager (drm)  A grid service that represents the external data resource manager  Binds to an existing edrm  Supports management operations such as start and stop  Mainly out of scope of DAIS. A place holder for interaction with other working groups

8 Data Resources External Data Resource (edr)  A data construct managed by the external data resource manager, for example, a database or a directory structure.  An external data resource manager may manage many external data resources Data Resource (dr)  A grid service that represents an external data resource  Represents the point of contact to the data structures managed by the edrm.  Exposes meta-data about the structure of the edr  Defines the the target for queries across the edr  Can act as a notification source for notifications associated with the edr  Is bound to existing or newly created edr  Has similarities with a data set. More of which later.

9 Data Sets External Data Set (eds)  Data logically separated from an external data resource manager Could be a snapshot (query) of a relational database or data generated by some process prior to being inserted into a database  Will be typed and identifiable Data Set (ds)  A service wrapper for the eds  Exposes meta data about the type, description, format of the eds  Immutable  Exposes simple data access operations depending on the type of data. getAllData, createIterator, getTuple, getFile, getByte, etc.  Can be moved while maintaining its handle and data identity  Can be copied or replicated while maintaining its data identity  Can be delivered to a data manager for persistence Query and update could be supported

10 Putting It Together Logical Artifact = Service ds eds dr edredrm drmdas External world DAIS world create bind/ create bind create bind/ create requester data request locateaccess data

11 Exploiting The Logical Artifacts: Data Sets dsdr edr das ds dr edr das query insert/update create movecopy reference launch GSH move service referencecreate copy service referencecreate target details target details Analyst1

12 Client-Server Interaction Patterns AG Q S + R A G Q + D S R C AG Q + U S AG Q S U P Retrieve Update/Insert A G Q S D C R AG Q + D S I P U Pipeline A G = P Q1 + D S1 U/R G = CS2 Q2 A G = PQ1 S1 U/R G = CS2 Q2 + D I 7. 8.

13 Examples of client-service interactions Discovery and creation of services

14 Examples of client-service interactions Asynchronous query processing and datasets

15 Examples of client-service interactions Updating from datasets

16 Example of performance estimation

17 SkyQuery Cross Match Query

18 Cross Match Estimation in DAIS

19 Cross Match in DAIS

20 Other topics and issues for DAIS Data provenance management Transaction management Fault tolerance Security, logging, auditing Supporting many concurrent users Establishing the identity and provenance of datasets Creating pipelines and other workflows Querying streams of data