1. VOX O rder-sensitive V iew Maintenance of Materialized X Query Views ER 2003 October 14 th 2003 Katica Dimitrova*, Maged El-Sayed and Elke Rundensteiner Worcester Polytechnic Institute *Now at Microsoft

2. 2 Motivation Views in general  Information integration  Access control, privacy,..etc  Data warehouses XML Views (EXTRA useful)  Information inter-portability  Crossing gaps between different data models Materialized Views  Fast access over complex views  Increased availability  Query optimization RDBXML Other Sources View View Definition Query

3. 3 View Maintaining Materialized Views Methods of view maintenance Recomputation  recompute view from scratch from base data View Source 1Source 2Sources 3..n View Definition Query update When sources are updated, materialized view may become inconsistent. Incremental view maintenance is usually cheaper than full recomputation. Incremental view maintenance  compute changes to view in response to changes to base sources update

4. 4 Goal Incrementally maintaining XQuery views Why is it a challenge?  XML features Hierarchical Optional elements Self-typed IDRefs Ordered  Expressiveness of XQuery language Complex operations: tagging, unnesting, aggregation,.. Expected large auxiliary information XML Source XML Source XML Source View View Definition XQuery

5. 5 Basics of VOX Approach: Algebraic General approaches to view maintenance  Algorithmic – Fixed procedure exists for fixed view type  Algebraic - Update propagation rules for each algebra operator and each update type XML Source XML Source XML Source XML View Update Algebra Tree XQuery Definition Operator D1 D2 Operator D1 Update D2 Update ExecutionView Maintenance time

6. 6 Example Insert element 55.48 into second book Bib.xml Data on the Web Data on the Web View Extent TCP/IP Illustrated 65.95 Advanced Programming in the Unix environment TCP/IP Illustrated 39.95 Data on the Web 65.95 Advanced Programming in the Unix environment TCP/IP Illustrated 39.95 Data on the Web 55.48 for $b in document("bib.xml")/bib/book where $b/price/text() < 60 return $b/title for $b in document("bib.xml")/bib/book where $b/price/text() < 60 return $b/title View Definition Query Bib.xml List all books that cost less than $60

7. 7 Background on XML Algebra XAT XQuery  XAT algebra tree [ZR02] XAT Operators:  XAT SQL Operators: Select, Project …  XAT XML Operators: Navigate Unnest, Navigate Collection, Tagger, Combine..  $s6, /book $b S “bib.xml” $s6  $b, title $col3  $b, price/text() $col5 T $col3 $col2 C $col2 T $col2 $col1 bib.xml  ($col5 < 60.0)  $col1 view for $b in document("bib.xml")/bib/book where $b/price/text() < 60 return $b/title for $b in document("bib.xml")/bib/book where $b/price/text() < 60 return $b/title

8. 8 Background on XML Algebra XAT – Data Model  $s6, /book $b S “bib.xml” $s6  $b, title $col3  $b, price/text() $col5 T $col3 $col2 C $col2 T $col2 $col1 bib.xml  ($col5 < 60.0)  $col1 view $col3$b Advanc.. 65.95 Advanc.. TCP/IP … Data on.. 39.95 Data on.. $col5$col3 65.95 Advanc.. TCP/IP … 39.95 Data on..  $b, price/text() $col5 XAT Data Model (XAT Table)  Order sensitive table of tuples  Columns denote user-specified or internally generated variable bindings  A cell in a tuple holds an XML node for a sequence of XML nodes The XAT algebra has ordered bag semantics InputOutput

9. 9 Order in XAT Context: View Maintenance $col5$col3 65.95 Advanced Prog… TCP/IP Illustrated 39.95 Data on the Web  ($col5 < 60.0) $col3 Data on the Web Non Order-sensitiveOrder-sensitive 55.48 $col3 Data on the Web TCP/IP Illustrated $col3 Data on the Web $col5$col3 65.95 Advanced Prog… TCP/IP Illustrated 39.95 Data on the Web $col3 Data on the Web  ($col5 < 60.0) 55.48 $col3 TCP/IP Illustrated Data on the Web

10. 10 node identity Use node identity Why?  Already present as concept in XQuery reference  Can be reference to base XML data set  Can encode structure and order Our Approach to Maintaining Order

11. 11 bib book price title price title b b.h b.n b.t b.h.r b.h.k b.n.m b.t.k b.t.r Lexicographical Keys: LexKeys price b.n.f Multi-level lexicographical keys Comparison b.h < b.t bab < bd.cc b.b < b.b.c Advantages  It is always possible to generate a key between two keys  The deletion of a LexKey in a sequence does not affect other LexKeys

12. 12 LexKeys - References to source XML nodes $col3$b b.h.rb.h b.n.mb.n b.t.rb.t $b b.h b.n b.t $b 65.95 Advanc.. TCP/IP … 39.95 Data on.. $col3$b Advanc.. 65.95 Advanc.. TCP/IP … TCP/IP … Data on.. 39.95 Data on..  $b, title $col3 bib.xml bib book price title price title b b.h b.n b.t b.h.r b.h.k b.n.m b.t.k b.t.r title Storage Manager

13. 13 LexKeys - References to constructed nodes $col3 b.n.m b.t.r $col2 y.c y.b Constructed Nodes SkeletonLexKey y.b y.c cheap_book b.t.r cheap_book b.n.m T $col3 $col2  $s6, /book $b S “bib.xml” $s6  $b, title $col3  $b, price/text() $col5 T $col3 $col2 C $col2 T $col2 $col1 bib.xml  ($col5 < 60.0)  $col1 view Storage Manager bib.xml bib book b b.h b.n b.t title b.t.r

14. 14 Order Among XAT Tuples Notion: designate order schema to XAT tables  Ordering by LexKeys in columns in order schema yields correct tuple order.  Comparison operation ‘<’ on tuples. $col3$b b.h.rb.h b.n.mb.n b.t.rb.t $b b.h b.n b.t  $b, title $col3 2 1 3 b.h < b.n < b.t 1 1

15. 15 Order Schema Computation Operatorop(R)Order Schema OS Q, Q = op(R) Tagger T pattern $col’ (R) OS R Calculated in a postorder traversal of the tree Schema Computation Rules

16. 16 Concept of overriding order Overriding Order (LexKey) Key (LexKey) LexKey with overriding order Node identity part, by default also represents order Optional, only represents order when present Notation: key [order] Examples  b.c.b [h]  b.c.b Order Among Nodes in a Cell Combine creates a collection in which nodes may be in order different then one encoded in node identity Most collections of XML nodes are in document order  Navigate Collection, XML Union, …

17. 17 The Impact of Using LexKeys on View Maintenance XML algebra now has (non- ordered) bag semantics Gained distributiveness with regard to bag union and difference Compact intermediate results $col5$col3$b b.h.k.mb.h.rb.h b.n.mb.n b.t.k.m b.t.rb.t  ($col5 < 60.0) $col3$b b.t.rb.t b.n.f.m $col3$b b.t.rb.t b.n.mb.n

18. 18 Update Propagation Strategy XML Source XML View Update XAT iup kk Storage Manager Rainbow Update XQuery

19. 19 Update Propagation Rules Operator R Q Update to R Update to Q ExecutionView Maintenance time Use distributiveness with regard to bag union Reuse rules from relational view maintenance for XAT SQL operators Provide rules for XAT XML operators

20. 20 Update Propagation Rules Example - Navigate Unnest on Insertion of Tuples Q old =  $col,path $col’ (R old ) R new =R old +  R Q new =  $col,path $col’ (R old +  R) = =  $col,path $col’ (R old ) +  $col,path $col’ (  R) = = Q old +  Q + represents bag union R Q  $col,path $col’ u (  R) u (  Q) ExecutionView Maintenance time  $col,path $col’ Propagate u(  Q)

21. 21 bib.xml Constructed XDOMs $col5$col3$btid b.h.k.mb.h.rb.h1 b.n.f.mb.n.mb.n2 b.t.k.mb.t.rb.t3  $s6, /book $b S “bib.xml” $s6  $b, title $col3  $b, price/text() $col5 T $col3 $col2 C $col2 T $col2 $col1  ($col5 < 60.0)  $col1 view bib book price title price title b b.h b.n b.t b.h.r b.h.k b.n.m b.t.k b.t.r $btidpid b.h11 b.n21 b.t31 65.95 39.95 b.h.k.m b.t.k.m tid 3 $b b.t SkeletonLexKey y.b cheap_book b.t.r $col1 x x result y.b[b.t] title View Maintenance Example  + b.n.f, book[b.n]/price[b.n.f] b u (  + b.n.f, book[b.n]/price[b.n.f] b, $s6, 1) u (  + b.n.f, price[b.n.f] b.n, $b, 2) u (  c, $col5, 2) |  c= b.n.f.m u (  s) |  s = (b.n, b.n.m) u (  s) |  s = (b.n, y.c ) u (  c, $col2, 1) |  c = y.c[b.n] u (  + y.c[b.n], result[1]/$col2 x, $col1, 1) u (  + b.n.f, price[b.n.f] b.n, $b, 2) Storage Manager Rainbow Insert element 55.48 into second book price b.n.f 55.48 b.n.f.m tid 3 2 y.c cheap_book b.n.m $b b.t b.n y.c[b.n]

22. 22 bib.xml Constructed XDOMs bib book price title price title b b.h b.n b.t b.h.r b.h.k b.n.m b.t.k b.t.r 65.95 39.95 b.h.k.m b.t.k.m SkeletonLexKey y.b cheap_book b.t.r x result y.b[b.t] title View Maintenance Example Storage Manager Rainbow Insert element 55.48 into second book price b.n.f 55.48 b.n.f.m y.c cheap_book b.n.m y.c[b.n] S “bib.xml” $s6 T $col2 $col1  $col1 view x $col1........ result y.b[b.t] y.c[b.n] cheap_book b.n.m TCP/IP Illustrated title cheap_book title Data on the Web

23. 23 637 elements of interest selectivity 50% Experimental Evaluation Basic performance comparisonVarying size of insert Implemented in Java on top of Rainbow system Experimental evaluation

24. 24 Related work Relational  [GMS93] Survey  [GL95] Algebraic approach to maintain relational views with duplicates  [BLT86], [CW91], [ZGHW95], [Q96], [MK00], [PSCP02]… Object-Oriented  [KR96] MultiView. Object algebra, exploit OO features like inheritance, path indexes.  [AFP02] Algebraic approach. Store OID-s rather then actual data. XML-like data models  [ZM98] Select-Project graph structured views as collections of objects.  [AMRVW98] Semistructured data model OEM, query language LOREL. Only atomic updates. Does not handle order.  [QLR02] Dynamic web data. Based on XPath. Maintains path index structure.  [LD00] Hierarchical semistructured data. View defined with WHAX-QL. Does not handle order.  [EWDR02] – Motivation for this work. Algebraic approach. Does not handle order. Large intermediate results.

25. 25 Conclusions Proposed order-encoding scheme that migrates XML algebra from ordered to non-ordered bag semantics Gave first solution to order-sensitive XQuery view maintenance  Handles core of XQuery  Handles complex updates Proved correctness of approach Implemented the solution within Rainbow Experimental evaluation confirms feasibility of solution

26. 26 For more information The Rainbow project http://davis.wpi.edu/dsrg/rainbow/ Related publications  K. Dimitrova, M. El-Sayed and E. Rundensteiner. Order-sensitive View Maintenance of Materialized XQuery Views. Technical Report WPI-CS-TR-03-17, May 2003.  M. El-Sayed, K. Dimitrova, E. Rundensteiner, Efficiently Supporting Order in XML Query Processing, WIDM'03, New Orleans, Nov.2003.  X. Zhang, K. Dimitrova, L. Wang, B. Pielech, L. Ding, B. Murphy, M. El-Sayed and E. Rundensteiner. RainbowII: Multi-XQuery Optimization Using Materialized XML Views. SIGMOD DEMO, Jun. 2003.  M. Sayed, L. Wang, L. Ding and E. Rundensteiner. An Algebraic Approach for Incremental Maintenance of Materialized XQuery Views. In Proceedings of WIDM02, page88, 2002.(.ps)ps  X. Zhang, B. Pielech and E. Rundensteiner. Honey, I Shrunk the Xquery!- An XML Algebra Optimization Approach. In Proceedings of WIDM02, 2002.  X. Zhang, M. Mulchandani, S. Christ, B. Murphy and E. Rundensteiner. Rainbow: Mapping-Driven XQuery Processing System. Proceeding of SIGMOD02, In Demo Session, page 614, 2002.

27. 27 Thank you !