Presentation is loading. Please wait.

Presentation is loading. Please wait.

Query Processing in the AquaLogic Data Services Platform Vinayak Borkar, Michael Carey, Dmitry Lychagin, Till Westmann, Daniel Engovatov, Nicola Onose.

Similar presentations


Presentation on theme: "Query Processing in the AquaLogic Data Services Platform Vinayak Borkar, Michael Carey, Dmitry Lychagin, Till Westmann, Daniel Engovatov, Nicola Onose."— Presentation transcript:

1 Query Processing in the AquaLogic Data Services Platform Vinayak Borkar, Michael Carey, Dmitry Lychagin, Till Westmann, Daniel Engovatov, Nicola Onose BEA Systems www.bea.com

2 (C) Copyright 2006, BEA Systems, Inc | 2 Data Is Everywhere Today Relational databases made things too easy Departmental vs. inter-galactic centralized databases Databases come in many flavors Relational: Oracle, DB2(s), SQL Server, MySQL, … Hangers-on: IMS, IDMS, VSAM, … Not all data is SQL-accessible Packaged applications: SAP, PeopleSoft, Siebel, Oracle, SalesForce, … Custom “homegrown” applications Files of various shapes and sizes (XML, non-XML) And the list goes on…

3 (C) Copyright 2006, BEA Systems, Inc | 3 Painful to Develop Applications No one “single view of X” for any X What data do I have about X? How do I stitch together the info I have? What else is X related to? No uniformity in source model or language Data about X is stored in many different formats Accessing or updating X involves many different APIs Manual coding of “distributed query plans” No reuse of artifacts Different access criteria or returned data  different access plans No model to help organize or find the artifacts anyway

4 (C) Copyright 2006, BEA Systems, Inc | 4 Agenda Why data services? Overview and Example The Query Processor Work in progress at BEA Summary and Q&A

5 (C) Copyright 2006, BEA Systems, Inc | 5 Overview JAVA/J2EE Web Services Business partners, Adapter,.Net LDAP J2EEExcel Java Functions Files XML, Flat Files Tables, views stored procedures, SQL Relational JAVA API JDBC/SQL Developer Tooling REPORTING Connectivity Client API Data Processing Engine WEB SERVICE WSDL JCA JMS Custom Access

6 (C) Copyright 2006, BEA Systems, Inc | 6 Example: Customer Profile Data Service CUSTOMER, ORDER CREDIT_CARD getRating(…) Order Info Credit Card Info Rating Info Customer Info

7 (C) Copyright 2006, BEA Systems, Inc | 7 Data Service – Design View

8 (C) Copyright 2006, BEA Systems, Inc | 8 Data Service – “Get All” Read Method kind="read" (::pragma function... kind="read"...::) declare function tns:getProfile() as element(ns0:PROFILE)* { for $CUSTOMER in db1:CUSTOMER() return { fn:data($CUSTOMER/CID) } { fn:data($CUSTOMER/LAST_NAME) } { db1:getORDER($CUSTOMER) } { db2:CREDIT_CARD()[CID eq $CUSTOMER/CID] } { fn:data(ws1:getRating( { data($CUSTOMER/LAST_NAME) } { data($CUSTOMER/SSN) } ) } };

9 (C) Copyright 2006, BEA Systems, Inc | 9 Data Service – Read & Navigate Methods kind="read" (::pragma function... kind="read"...::) declare function tns:getProfileByID($id as xs:string) as element(ns0:PROFILE)* { tns:getProfile()[CID eq $id] };... kind="navigate" (::pragma function... kind="navigate"...::) declare function tns:getCOMPLAINTs($arg as element(ns0:PROFILE)) as element(ns8:COMPLAINT)* { db3:COMPLAINT()[CID eq $arg/CID] };...

10 (C) Copyright 2006, BEA Systems, Inc | 10 Agenda Why data services? Overview and Example The Query Processor Work in progress at BEA Summary and Q&A

11 (C) Copyright 2006, BEA Systems, Inc | 11 Query Processor Overview

12 (C) Copyright 2006, BEA Systems, Inc | 12 Efficient processing Avoid unnecessary work Function inlining (view unfolding) Push work to the sources Queryable sources can do some of our work

13 (C) Copyright 2006, BEA Systems, Inc | 13 Optimization: Function Inlining Example: This fragment let $x := {$name} … return fn:data($x/LAST_NAME) can be replaced by $name But we need to: maintain structural type information for compilation extend “preserve” mode for runtime

14 (C) Copyright 2006, BEA Systems, Inc | 14 Pushdown: Overview SQL Translator tries to “swallow” as much as possible. Make translation easy Remove unnecessary functions (by inlining) Translation to joins and grouping Split sorting and grouping Make sources do as much as possible DBMS specific code Maximize Pushdown Inverse functions

15 (C) Copyright 2006, BEA Systems, Inc | 15 Pushdown: Preparation Translation to joins and grouping Split sorting and grouping for $CUSTOMER in db1:CUSTOMER() return { fn:data($CUSTOMER/CID) } … { db1:getORDER($CUSTOMER) } { db2:CREDIT_CARD()[CID eq $CUSTOMER/CID] } …

16 (C) Copyright 2006, BEA Systems, Inc | 16 Pushdown: Inverse Functions I Example fragment of tns:getProfile: {fn:data($CUSTOMER/CID)} { fn:data($CUSTOMER/LAST_NAME) } {int2date($CUSTOMER/SINCE)}... used in this query for $c in tns:getProfile() where $c/SINCE gt $start return $c yields (after inlining) for $c1 in ns3:CUSTOMER() where int2date($c1/SINCE) gt $start return...

17 (C) Copyright 2006, BEA Systems, Inc | 17 Pushdown: Inverse Functions II Register Inverse function date2int for int2date Transformation rule ( gt, int2date )  gt-intfromdate with declare function gt-intfromdate($x1 as xs:dateTime, $x2 as xs:dateTime) as xs:boolean?{ date2int($x1) gt date2int($x2) }; Now we can rewrite the query into for $c1 in ns3:CUSTOMER() where $c1/SINCE gt ns1:date2int($start) return... can be pushed as SELECT * FROM "CUSTOMER" t1 WHERE (t1."SINCE" > ?)

18 (C) Copyright 2006, BEA Systems, Inc | 18 Optimization: PP-k Join Parameter Passing in chunks of k Prerequisites Distributed join Right side is a relational source Idea: relational source can partition its content Steps Read k items from the left into L Select all items from the right that match any of the items in L into R Join L and R in the middleware Repeat until the left is exhausted Benefit: excellent trade-off between Memory footprint in the middleware Roundtrip overhead imposed by the data source k = 20

19 (C) Copyright 2006, BEA Systems, Inc | 19 Example: “Get All” Read Method Revisited (::pragma function... kind="read"...::) declare function tns:getProfile() as element(ns0:PROFILE)* { for $CUSTOMER in db1:CUSTOMER() return { fn:data($CUSTOMER/CID) } { fn:data($CUSTOMER/LAST_NAME) } { db1:getORDER($CUSTOMER) } { db2:CREDIT_CARD()[CID eq $CUSTOMER/CID] } { fn:data(ws1:getRating( { data($CUSTOMER/LAST_NAME) } { data($CUSTOMER/SSN) } ) } };

20 (C) Copyright 2006, BEA Systems, Inc | 20 Evaluation Plan, Example 1 (getProfile)

21 (C) Copyright 2006, BEA Systems, Inc | 21 Evaluation Plan, Example 2 (query getProfile)

22 (C) Copyright 2006, BEA Systems, Inc | 22 Agenda Why data services? Overview and Example The Query Processor Work in progress at BEA Summary and Q&A

23 (C) Copyright 2006, BEA Systems, Inc | 23 Some ALDSP Work in Progress Native JDBC/SQL92 support (available on Sep 15) Bilingual engine for efficient reporting/BI tool access Support for compensating transactions Extend update facility to support safe non-XA updates (sagas) XQuery Update support (as well as XQueryP) Goal: no Java coding for many Web service use cases

24 (C) Copyright 2006, BEA Systems, Inc | 24 Agenda Why data services? Overview and Example The Query Processor Work in progress at BEA Summary and Q&A

25 (C) Copyright 2006, BEA Systems, Inc | 25 Summary Covered here Why Data Services? How are Data Services used? Some techniques for efficient evaluation In the paper XQuery extensions Runtime architecture Caches Updates Security


Download ppt "Query Processing in the AquaLogic Data Services Platform Vinayak Borkar, Michael Carey, Dmitry Lychagin, Till Westmann, Daniel Engovatov, Nicola Onose."

Similar presentations


Ads by Google