1 Ling Wang Advisor: Elke A. Rundensteiner Co-Advisor: Kathi Fisler Updating XQuery Views Published over Relational Data

2 Outline Motivation Related Work Background: XML, XML Views, XQuery Update Extension Problem Definition: - Characterize the XML View Update Problem (XVUP) - Correct Translatability - Classify subproblems: RUP, PUP Update Strategy for PUP Rainfall System Experimental Study Contributions Conclusion

3 XML is a standard for exchanging data between web applications But RDBMS is mature data management technology: - Reliable persistent storage technique - Mature query optimization technique XML Management Systems using Relational technology - SilkRoute (AT&T), XPERANTO (IBM), RAINBOW (WPI) - Features: # provide reliable persistent storage # support XML view mechanism for XML data publishing # support queries over XML views Systems must support update to become viable system Motivation

4 View update problem in Relational databases is hard - What is a correct translation? - How to eliminate ambiguity in translation? XML view update offers new challenges - Mappings: Data model mapping: XML vs. Relational Query language mapping: XQuery vs. SQL - Consistency : Constraints in different schemas such as XML schema, Relational schema, XML view schema XML View Update RDBMS View Query XML View XQuery update SQL Update RDBMS Default View eg: Schema reducing (IDREF), Schema increasing (Duplicate)

5 Related Work 1.Umeshwar Dayal, Philip A. Bernstein, On the Updatability of Relational Views, IEEE First work, describe criteria of correct update translation. 2.F Bancilhon and N. Spyratos, Update Semantics of Relational Views, ACM Transactions on Database Systems, 1984 Complementary theory to eliminate ambiguous in update translation. 3.A. M. Keller, Barsalou, Siambela and Wiederhold, Updating Relational Databases through Object-Based Views, SIGMOD 1991 View update problem in Object-base views. 4.Tatarinov Ives, Alon Halevy and Daniel Weld, Updating XML, SIGMOD 2001 XQuery update extension, XML view update performance

6 Where we are Motivation Related Work Background: XML Views, XQuery Update Extension Problem Definition: - Characterize the XML View Update Problem (XVUP) - Correct Translatability - Classify subproblems: RUP, PUP Update Strategy for PUP Rainfall System Experimental Study Contributions Conclusion

7 XML Schema ……

8 XML document Bib.xml Addison-Wesley Boston A clear and detailed discussion of UNIX programming Data on the Web Serge Abiteboul Peter Buneman Dan Suciu Morgan Kaufmann Publishers New York A very good discussion of semi-structured database systems and XML TCP/IP Illustrated W. Stevens Addison-Wesley San Francisco One of the best books on TCP/IP Programming in Unix Bram Stoker

9 Example Database ( Inlining) IIDPIDaname W.Stevens Bram Stoker Serge Abiteboul Peter Buneman Dan Suciu IIDPIDbookidtitleauthor_IIDyear TCP/IP Illustrated Programming in Unix Data on the Web author book IIDpnamelocation 1.0 Addison-Wesley San Francisco 2.0Addison-WesleyBoston 3.0Morgan Kaufman publishers New York publisher IIDPIDreview 1.0 One of the best books on TCP/IP. 2.0 A clear and detailed discussion of UNIX programming. 3.0 A very good discussion of semi- structured database systems and XML. review Legend: Primary Key Unique Key Non Key RDBMS View Query XML View XQuery update SQL Update RDBMS Default View

10 Default View TCP/IP Illustrated W. Stevens Addison-Wesley SanFrancisco One of the best books on TCP/IP Bridging the gap between Relational database and XML data models - Show database structure, 1:1 mapping RDBMS View Query XML View XQuery update SQL Update RDBMS Default View

11 XML View FOR $book in document("default.xml")/book/row RETURN{ $book/title/text(), FOR $aname in document("default.xml")/author/row WHERE $book/author_IID = $aname/PID RETURN{ $aname/aname/text() } } - Information hiding - User specific data wrapping TCP/IP Illustrated W. Stevens Programming in Unix Bram Stoker Data on the Web Serge Abiteboul Peter Buneman Dan Suciu View Query Virtual XML View RDBMS View Query XML View XQuery update SQL Update RDBMS Default View

12 XQuery Update Grammar FOR $binding1 IN Xpath-expr,….. LET $binding := Xpath-expr,… WHERE predicate1,….. updateOp,…… Where updateOp is defined as : UPDATE $binding {subOp {, subOp}* } and subOp is : DELETE $child | INSERT ( $bind [BEFORE | AFTER $child] | new_attribute(name, value) | new_ref(name, value) | content [BEFORE | AFTER $child] ) | REPLACE $child WITH ( new_attribute(name, value) | new_ref(name, value) | content ) | FOR $sub_binding IN Xpath-subexpr,….. WHERE predicate1,………. updateOp. - XQuery does not support update operations - Update extension for XQuery language proposed in [SIGMOD2001] - FLWU Expression RDBMS View Query XML View XQuery update SQL Update RDBMS Default View

13 FOR $book IN document(“bib.xml")/book LET$author:=$book/author WHERE $book/title = “TCP/IP Illustrated” UPDATE $author{ INSERT "Peter Naughton " } Insert update Update XQuery example Updated view TCP/IP Illustrated W. Stevens "Peter Naughton " Programming in Unix Bram Stoker Data on the Web Serge Abiteboul Peter Buneman Dan Suciu RDBMS View Query XML View XQuery update SQL Update RDBMS Default View

14 Where we are Motivation Related Work Background: XML Views, XQuery Update Extension Problem Definition: - Characterize the XML View Update Problem (XVUP) - Correct Translatability - Classify subproblems: RUP, PUP Update Strategy for PUP Rainfall System Experimental Study Contributions Conclusion

15 General Characterization of XVUP Information Modification Instance Language View Query RDB Content RDB Schema Integrity Constraints (Key, FK) Local Constraints (Not Null, domain) BCNF / 3NF / 2NF Acyclic Dependency / Cyclic Dependency Duplicates ( Aggregation ) ( Recursion ) Hierarchy Consistency Key Exposition Non Correlation predicates attribute exposition Correlation predicates attributes exposition Nested queries Deletion Insertion Replacement Rename Order Set of each Group Update ( Schema Change ) Assumption Exploring space

16 Schema Mapping in XVUP Constraints in Extracted XML View Schema Cardinality Hierarchy from view query Duplication Constraints in XML Schema/DTD Domain of element/attributes Key Constraints Referential Integrity Constraints (Foreign Key) Optional element Nillable attributes Cardinality constrains Hierarchy Choice Inclusion dependency (ID/IDREF) Sequence Constraints in Relational Database schema Domain Constraints of attribute in relation Primary key / unique key of table Foreign key between tables Attributes is Null / Not null Table definition Global Constraints Constraints in Filtered XML View Schema Domain Cardinality Null/Not null Hierarchy ? ? XML View Schema ESchemaFSchema Loading View Query Problem Definition: - Characterize the XVUP - Schema mapping in XVUP - Classify subproblems: RUP, PUP - Update operation - Correct translatability

17 Example of FSchema vs. ESchema FOR $book in document("default.xml")/book/row RETURN{ $book/bookid/text(), $book/title/text(), FOR $aname in document("default.xml")/author/row WHERE $book/author_IID = $aname/PID RETURN{ $aname/aname/text() } } IIDPIDaname W.Stevens IIDPIDbookidtitleauthor_IIDyear TCP/IP Illustrated author book Fschema: bookid (Not Null) title (Varchar(256)) Eschema: Hierarchy: book = parent(bookid) Cardinality: (book:aname) = (1:n) Relational database View Query

18 Partition for XVUP Problem Space Publishing-based Update Problem (PUP) - General problem ---- updating freely constructed XQuery view. - Touching all features in Language dimension. Round-trip Update Problem (RUP) - Simplest special problem. - View has same schema as original XML document - No abnormal case exists. RDBMS View Query XML View RDBMS XML Doc+Schema 1 2 RUP PUP Problem Definition: - Characterize the XVUP - Schema mapping in XVUP - Classify subproblems: RUP, PUP - Update operation - Correct translatability

19 XML View Update Operation Complete vs. Partial update - Complete ---- insert/delete/replace a fully specified view-element instance - Partial ---- insert/delete/replace a component of view-element instance Valid vs. Invalid update - Valid ---- Never against any constraints in ESchema, Fschema (domain+null). Update translation. - Invalid ---- Against Constraints. Rejected TCP/IP Illustrated W. Stevens Programming in Unix Bram Stoker Data on the Web Serge Abiteboul Peter Buneman Dan Suciu Complete Partial Problem Definition: - Characterize the XVUP - Schema mapping in XVUP - Classify subproblems: RUP, PUP - Update operation - Correct translatability

20 Correct Update Translatability No side effect Only desired update is performed on view. One step changes - Each database tuple affected by at most one step of update operation - Implications: No order between update operations Could affect same table several times Minimal changes - No valid translation is subset of current translation - No extraneous updates Replacement cannot be simplified - Two replaces could get same result, pick simple one - Replace minimum attribute set / non-key replacement No insert-delete pairs - Replace is cheaper than Insert/Delete pair Problem Definition: - Characterize the XVUP - Schema mapping in XVUP - Classify subproblems: RUP, PUP - Update operation - Correct translatability

21 Where we are Motivation Related Work Background Problem Definition Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP Rainfall System Experiment Study Contributions Conclusion

22 Problem for PUP Information Modification Instance Language Duplicates ( Aggregation ) ( Recursion ) Hierarchy Consistency Key Exposition Non Correlation predicates attribute exposition Correlation predicates attributes exposition Nested queries Assumption Exploring space Exposition feature Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP

23 Exposition Completeness Key Exposition - Primary key of underlying relation has to be exposed - except automatic generated key (implication: user has right to update it) Non Correlation predicates attribute exposition (select condition) - variable involved in predicates has to be exposed eg: $book/bookid = “98004” bookid has to be exposed in view result Correlation predicates attribute exposition (join condition) - variable involved in predicates has to be exposed eg: $book/authorid = $author/id then authorid in book table, id in author table have to be exposed Complete Exposition Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP

24 Construction Consistency IIDPIDbookidtitleauthor_IIDyear TCP/IP Illustrated Advanced Programming in the Unix environment Data on the Web book IIDpnamelocation 1.0 Addison-Wesley San Francisco 2.0Addison-WesleyBoston 3.0Morgan Kaufman publishers New York publisher Reason: freely construct the view query Confliction between FSchema and ESchema View structure could be inconsistent with RDB, causing side-effect in update translation. Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP

25 FOR $book IN document("default.xml")/book/row, $publisher IN document("default.xml")/publisher/row WHERE $book/author_IID = $publisher/IID RETURN $book/bookid, $book/title, $publisher/pname TCP/IP Illustrated Addison-Wesley …… Construction Consistency FOR $book IN document("default.xml")/book/row, $publlisher IN document("default.xml")/publisher/row WHERE $book/IID = $publisher/IID RETURN $publisher/pname, $book/bookid, $book/title Addison-Wesley TCP/IP Illustrated …… View Query Example Consistent View Query Example Inconsistent Delete Side Effect Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP

26 How to find inconsistent construction? Implicit hierarchy in Relational context: - Table vs. Attribute - Key vs. Foreign-Key bookidtitleyearpnamelocationreview BOOK bookidauthoridname AUTHOR Then, book is parent of all its attributes, book is parent of author Hierarchy in View context - Return clause in FLWR expression - Variable binding in FOR clause View Schema Graph (VSG) to describe hierarchy in FSchema Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP View Relationship Graph (VRG) to describe hierarchy in ESchema

27 bib Publisher_info book book/row/bookid book/row/title publisher/row/pname book book/row/bookidbook/row/titlepublisher Publisher/row/pname Hierarchy in relational context: View Relationship Graph Hierarchy in view context: View Schema Graph Reducing Operator: - Rename - Add - Delete bib book book/row/bookid book/row/title Publisher_info Publisher/row/pname Rename

28 Duplicate - Two vertexes in VSG are exposed from same relational attribute. - Partial updates touching duplicate elements are not translatable. why? Cause ambiguous/inconsistent effect on underlying relation. book publisher author book/row/title update book/row/title author/row/aname publisher/row/pname Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP

29 View FeaturesUpdate Type Exposition complete ConsistencyNo Duplication Complete Update Partial Update YYYYY YYNYCase 1 YNYYCase 2 YNNYCase 3 NYYNN NYNNN NNYNN NNNNN View Updatability in PUP Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP Well-formed View

30 Case study in PUP Case 0: Well-formed View Complete Exposition + Consistent Construction + No Duplication Case 1: Complete Exposition + Consistent + Duplication Partial update + touch one side of duplication is not translatable Case 2: Complete Exposition + In-Consistent + No-Duplication sub-tree rooted at inconsistent edge is not updatable Case 3: Complete Exposition + In-Consistent + Duplication case 1  case 2 Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP

31 Round-Trip Update Problem (RUP) - View is the same structure and semantics as the original XML document. - RUP falls into Well-formed View. - Any Valid Updates on it are always Translatable TCP/IP Illustrated W. Stevens Addison-Wesley San Francisco One of the best books on TCP/IP. … IIDPIDbookidtitleauthor_IIDyear book IIDPIDreview IIDPIDaname author IIDpnamelocation publisher Complete exposition? YES Construction consistency? YES NO Duplication? YES Update Strategy for PUP Exposition completeness Construction consistency Duplicate View updatability in PUP

32 Where we are Motivation Related Work Background Problem Definition - Characterize the XML View Update Problem (XVUP) - Correct Translatability - Classify subproblems: RUP, PUP Update Strategy for PUP Rainfall System Experiment Study Contributions Conslusion

33 XAT Rewriter View Composer DB2 View Query Parsed Tree Multiple SQL updates SQL Generator User Query SQL XAT Executor XAT XAT Generator XQuery Parser View Query Oracle XQuery View Manager SQL-ServerSybase RDBMS XAT Information Collecter View Analyzer Valid Update Checker Update Decomposer Update Translator Update Propagator Meta -Data XAT View Query XAT Materialized data Process Step Process Flow Data Flow RDBMS Legend Rainfall Rainbow Query Engine Result XML

34 Function updateDecomposition (XAT t) begin XATleaves  get all leave node of XAT t resultUpdate  new array of RelationalUpdate for ( all node n in XATleaves ) up  new RelationalUpdate up.updateType  same type as root node type of XAT t while ( n is not null ) opUpdateDecomp(node, up) n  parent of node n decideUpdateType (up) put up into resultUpdate distinctResultUpdate(resultUpdate) End Function decideUpdateType (RelationalUpdate up) begin if ( up.updateType is insert or replace ) do nothing else if ( up.updateType is delete ) if ( up.updateColumns include all attributes of relation up.tableName ) up.updateType  delete else up.updateType  replace end Rainfall Update System Fundamental concept Update decomposition Update propagation Update Decomposition Algorithm

35 Update Decomposition Tagger Source Join Nav Source Function opUpdateDecomp( XATNode n, RelationalUpdate up) Begin if ( n is a Navigate node ) up.tableName  get table name from n if it has one else if ( n is a Select node or a Join node ) up.whereClause  condition in predicate of node n else if ( n is a Tagger node ) if ( up.updateType is delete ) if ( n.DOMpattern matches DOM pattern of the element node to be inserted ) up.updateColum.columnName  extract names of attributes from Tagger pattern if ( up.updateType is insert ) if ( n.DOMpattern matches DOM pattern of the element node to be inserted ) up.updateColumn.columnName  extract names of attributes from Tagger pattern up.updateColumn.columnValue  extract attribute values from pattern of inserted element if ( up.updateType is replace) if ( n.DOM pattern matches DOM pattern of the element node to replacing ) up.updateColum.columnName  extract names of attributes from Tagger pattern up.updateColumn.oldColumnValue  extract old values from relational database up.updateColumn.columnValue  extract values from pattern of the replacing element else do nothing return up end

36 Fundamental ---- connection Ownership connection Subset connectionReferencing connection X1 X2 X1=UK/PK(R1), X2  PK(R2) X1=UK/PK(R1), X2=PK(R2) X1=UK/PK(R1), X2  NK/UK(R2), Cardinality 1:n 1:[0,1] 1:n Representation R1 R2 R1 R2 R1 R2 R We divide Foreign key as three types: Inner-going Outer-going Rainfall Update System Fundamental concept Update decomposition Update propagation FK

37 Book Author Review State Publisher Prices PhoneBook 1. Delete 2. Delete 3. Delete 4. Delete 5. Replace Example for delete propagation -Dependency Island - Referencing peninsula Rainfall Update System Fundamental concept Update decomposition Update propagation

38 Example for insert propagation Book Author Review State Publisher Prices PhoneBook 1. Extend 2. (C1) Insert (C2) Reject (C3) Replace 3. Checking Rainfall Update System Fundamental concept Update decomposition Update propagation

39 Where we are Motivation Related Work Background Problem Definition - Characterize the XML View Update Problem (XVUP) - Correct Translatability - Classify subproblems: RUP, PUP Update Strategy for PUP Rainfall System Experiment Study Contributions Conclusion

40 Updating vs. Re-loading Note: (1) Loading strategy: shared inlining. (2) View is same complexity with document (RUP)

41 Translation for different update types Note: (1) Same View between update types (2) Update is same granularity

42 Performance among different translation steps Note: (1) Database size: number of books= 800. (2) Same view between different update types (3) Update is translatable (4) Update has same granularity

43 Where we are Motivation Related Work Background Problem Definition - Characterize the XML View Update Problem (XVUP) - Correct Translatability - Classify subproblems: RUP, PUP Update Strategy for PUP Rainfall System Experiment Study Contributions Conclusion

44 Contributions Characterize and propose a classification of the XML view update problem, and describe the updatability of each class. Present a decomposition-based update translation algorithm that correctly translates XQuery updates on views into a set of SQL-level updates. Implement Update System Framework (Rainfall) to implement the proposed strategy. Provide experimental results studying the performance of update strategies.

45 Conclusion We study the view update problem in XML context and provide solution for solving this problem Future work. - Ordersensitive view update - Batch update processing - Performance in update translation - Choose trigger for query plan optimization - Use index speeding up path expression evaluation - Use materialized view XAT to improve performance - re-write and optimize XAT - How to deal with schema changes

46 Thanks Advisor: Prof. Elke A. Rundensteiner. Co-advisor: Prof. Kathi Fisler. Group member: Mukesh Mulchandani in Rainfall system implementation. Xin Zhang and Rainbow group members in base system supporting.

47 Appendix Algorithm used in our Update Strategy

48 Dependency Island (DI) of root relation R - Rooted at R. - Maximal sub-tree. - All inner-going ownership and subset connections of R. Referencing Peninsula (RP) of root relation R - A relation Rj - Directly connected to any relation of dependency island R k via Reference connection Rk Rj Referenced Continent( RC) of root relation R - Rooted at R. - Maximal sub-tree. - All outer-going ownership / subset / reference connections of R. Dependency Island /Reference Peninsula R DI RP RC Rainfall Update System Fundamental concept Update decomposition Update propagation

49 View Mapping Graph (VMG) for running example book authorpublisherreview Rainfall Update System Fundamental concept Update decomposition Update propagation

50 Delete Propagation Assuming update is to delete a tuple t from relation R. (1)Isolate the dependency island DI of r. (2)For each relation in DI of r, delete all tuples dependent on t. (3)For each reference peninsula attached to DI of r, perform a replacement on the foreign key of each matching tuple. (4) For relations in DI of r that has dependency island, deletion is propagated( repeatedly, if necessary) to their DI. (5) For those relations involved in deletion, replacement is executed on their RP. Rainfall Update System Fundamental concept Update decomposition Update propagation

51 Insertion Propagation Assuming update is to insert a tuple t into relation R. (1)Extend the view tuple t as t' with values for the attributes that have been exposed out in the view definition. (2) Case 1: An identical tuple exists in R, reject the update. (3) Case 2: If t' does not match any key of underlying database relation, perform an insertion in R. Further, for RC of R, a check-insertion operation is done. (4)Case 3: If t' matches the key of an existing tuple, but some values for non-key attributes differ. If R has dependency island, reject the update, otherwise, performe replacement of t with t'. Rainfall Update System Fundamental concept Update decomposition Update propagation