1. SDPL 2002Notes 9: XQuery1 9 Querying XML Data and Documents n XQuery, W3C XML Query Language –"work in progress", Working Draft, 30 April 2002 –joint work by XML Query and XSL Working Groups »together with XPath 2.0 (  XPath 1.0 and XQuery) –influenced by many research groups and query languages »Quilt, XPath, XQL, XML-QL, SQL, OQL, Lorel,... –Goal: a query language applicable to any XML- represented data: both documents and databases

2. SDPL 2002Notes 9: XQuery2 Functional Requirements (1) n Support operations (selection, projection, aggregation, sorting, etc.) on all data types: –Choose parts of data based on content or structure –Also operations on document hierarchy and order n Structural preservation and transformation: –Preserve the relative hierarchy and sequence of input document structures in the query results –Transform XML structures and create new XML structures n Combining and joining: –Combine related information from different parts of a given document or from multiple documents

3. SDPL 2002Notes 9: XQuery3 Functional Requirements (2) n Cross-references: –must be able to traverse intra- and inter-document references n Closure property: –The result of an XML query is also XML (usually not a valid document, but a well-formed document fragment) –The results of a query can be used as input to another query n Extensibility: –The query language should support the use of externally defined functions on all data types of the data model

4. SDPL 2002Notes 9: XQuery4 XQuery in a Nutshell n Functional expression language n Strongly-typed: (XML Schema) types may be assigned to expressions statically n Includes XPath 2.0 (says Draft, but not all XPath axes included!) –XQuery 1.0 and XPath 2.0 share extensive functionality: »XQuery 1.0 and XPath 2.0 Functions and Operators, WD 30/4/2002 n Roughly: XQuery  XPath' + XSLT' + SQL'

5. SDPL 2002Notes 9: XQuery5 XQuery: Basics n A query is represented as an expression n Expressions operate on and return sequences of –atomic values (of XML Schema simple types) and –nodes (same 7 node types as in XPath/XSLT) –an item  a singleton sequence –sequences are flat: no sequences as items »(1, (2, 3), (), 1) = (1, 2, 3, 1) –sequences are ordered, and can contain duplicates n Unlimited combination of expressions, often with automatic type conversions (e.g. for arithmetics)

6. SDPL 2002Notes 9: XQuery6 XQuery: Basics n Query applied to an input sequence, accessed through functions... –input() »implementation-dependent input sequence –collection(" URI ") »sequence of nodes from URI –document(" URI ") »root of the XML document available at URI »this also in XSLT

7. SDPL 2002Notes 9: XQuery7 Some Central XQuery Expressions n path expressions n creation and combination of sequences (union, intersect, except) n element constructors (  XSLT templates) n FLWR (”flower”; for-let-where-return) expressions n sort expressions n quantified expressions (some/every … satisfies …) n testing and modifying datatypes; validation

8. SDPL 2002Notes 9: XQuery8 Path Expressions n Extend the syntax of XPath: [/]Expr/… /Expr –arbitrary (node-sequence-valued) expressions as steps –only 6 (of 13) axes: child, descendant, attribute, self, descendant-or-self, parent »combined with order comparison operators (precedes, follows) sufficient for queries (?) –produce ordered sequence of nodes in document order, without duplicate nodes –dereferencing for ID/IDREF values: Figure(s) referenced by refid attribute of the first figref child: figref[1]/@refid => figure

9. SDPL 2002Notes 9: XQuery9 Element Constructors n Similar to XSLT templates: –start and end tag enclosing the content –literal fragments written directly, expressions enclosed in braces { and } n often used inside another expression that binds variables used in the element constructor –See next

10. SDPL 2002Notes 9: XQuery10 Example Generate an emp element containing an empid attribute and nested name and job elements; values of the attribute and nested elements given in variables $id, $n, and $j : Generate an emp element containing an empid attribute and nested name and job elements; values of the attribute and nested elements given in variables $id, $n, and $j : {$n} {$n} {$j} {$j} </emp>

11. SDPL 2002Notes 9: XQuery11 FLWR ("flower") Expressions n Constructed from for, let, where and return clauses (~SQL select-from-where) n Form: ( F orClause | LetClause)+ WhereClause? "return" Expr n a FLWR expression binds values to one or more variables, and uses these variables to construct a result (in general, an ordered sequence of nodes)

12. SDPL 2002Notes 9: XQuery12 Flow of data in a FLWR expression

13. SDPL 2002Notes 9: XQuery13 for clauses for $V 1 in Exp 1 (, …) associates each variable V i with an expression Exp i that returns a list of items (e.g. a path expression) for $V 1 in Exp 1 (, …) associates each variable V i with an expression Exp i that returns a list of items (e.g. a path expression) n Result: list of tuples, each containing a binding for each of the variables n can be though of as loops iterating over the items returned by respective expressions

14. SDPL 2002Notes 9: XQuery14 let clauses n let also binds one or more variables to one or more expressions –each variable to the entire value of its expression (without iterating through items of the sequence) –results in binding a single sequence for each variable n Compare: –for $x in /library/book  many bindings (books) –let $x := /library/book  single binding (to a sequence of books)

15. SDPL 2002Notes 9: XQuery15 for/let clauses n A FLWR expr may contain several fors and lets –each of these clauses may refere to variables bound in previous clauses n the result of the for/let sequence: –an ordered list of tuples of bound variables –number of tuples = product of the cardinalities of the node-lists returned by the expressions in the for clauses

16. SDPL 2002Notes 9: XQuery16 for/let clauses - Example 1 for $i in (1,2), $j in (3,4) return {$i} {$j} for $i in (1,2), $j in (3,4) return {$i} {$j} Result:<tuple><i>1</i><j>3</j></tuple><tuple><i>1</i><j>4</j></tuple><tuple><i>2</i><j>3</j></tuple><tuple><i>2</i><j>4</j></tuple>

17. SDPL 2002Notes 9: XQuery17 for/let clauses - Example 2 let $s := (,, ) return {$s} return {$s} Result:<out> </out>

18. SDPL 2002Notes 9: XQuery18 where clause n binding tuples generated by for and let clauses filtered by an optional where clause –those with a true condition are used to instantiate the return clause the where clause may contain several predicates connected by and, or, and not() the where clause may contain several predicates connected by and, or, and not() –usually refer to the bound variables –sequences treated as Booleans similarly to node-lists in XPath: empty ~ false; non-empty ~ true

19. SDPL 2002Notes 9: XQuery19 where clause n Variables bound by a for clause represent a single item  scalar predicates, e.g. $p/color = ”Red”  scalar predicates, e.g. $p/color = ”Red” n Variables bound by a let clause may represent lists of nodes  list-oriented predicates, e.g. avg($p/price) > 100 –a number of aggregation functions available: avg(), sum(), count(), max(), min() (also in XPath 1.0)

20. SDPL 2002Notes 9: XQuery20 return clause n The return clause generates the output of the FLWR expression n instantiated once for each binding tuple n often contains element constuctors, references to bound variables, and nested subexpressions

21. SDPL 2002Notes 9: XQuery21 Examples (from "XML Query Use Cases") Assume: a document named ” bib.xml ” containing of a list of book s: + + Assume: a document named ” bib.xml ” containing of a list of book s: + +

22. SDPL 2002Notes 9: XQuery22 { { } } List the titles of books published by Morgan Kaufmann after 1998 for $b in document(”bib.xml”)//book where $b/publisher = ”Morgan Kaufmann” and $b/year gt 1998 and $b/year gt 1998 return return {$b/title}

23. SDPL 2002Notes 9: XQuery23 Result could be... <recent-MK-books> TCP/IP Illustrated TCP/IP Illustrated Advanced Programming in the Unix environment Advanced Programming in the Unix environment </recent-MK-books>

24. SDPL 2002Notes 9: XQuery24 List each publisher with the average price of its books for $p in distinct-value( document(”bib.xml”)//publisher) let $a := avg(document(”bib.xml”)/book[ publisher = $p]/price) return { { {$p/text()}, {$p/text()}, {$a} {$a} } }

25. SDPL 2002Notes 9: XQuery25 Invert the book-list structure { {-- group books by authors --} { {-- group books by authors --} for $a in distinct-values( document(”bib.xml”)//author ) for $a in distinct-values( document(”bib.xml”)//author ) return return {$a/text()} {$a/text()} for $b in document(”bib.xml”)//book[ for $b in document(”bib.xml”)//book[ author = $a] return $b/title return $b/title } }

26. SDPL 2002Notes 9: XQuery26 Make an alphabetic list of publishers; for each publisher, list books (title & price) in descending order of price for $p in distinct-values( document(”bib.xml”)//publisher ) return {$p/text()} {$p/text()} {for $b in document(”bib.xml”)/book[ {for $b in document(”bib.xml”)/book[ publisher = $p] return {$b/title, return {$b/title, $b/price} $b/price} sortby(price descending)} sortby(price descending)} sortby(name) sortby(name)

27. SDPL 2002Notes 9: XQuery27 Queries on Document Order n Operators precedes and follows to express conditions on document order The next example involves a surgical report with elements procedure, incision and anesthesia The next example involves a surgical report with elements procedure, incision and anesthesia n return a "critical sequence" that contains all contents between the 1st and 2nd incisions of the 1st procedure

28. SDPL 2002Notes 9: XQuery28 Computing a "critical sequence" { { let $p := //procedure[1] let $p := //procedure[1] for $n in $p/node() for $n in $p/node() where $n follows ($p//incision)[1]and $n precedes ($p//incision)[2] where $n follows ($p//incision)[1]and $n precedes ($p//incision)[2] return $n return $n } }

29. SDPL 2002Notes 9: XQuery29 Filtering n Function filter(Expr) –in XQuery core function library n returns shallow copies of the nodes selected by Expr –mutual order and hierarchy btw nodes the same as in input sequence

30. SDPL 2002Notes 9: XQuery30 Hierarchical Filtering n Only selected nodes remain; other levels are collapsed

31. SDPL 2002Notes 9: XQuery31 Table of contents for”cookbook.xml”, consisting of section titles (at any level of nesting) let $b := document(”cookbook.xml”) return { { filter($b//(section | filter($b//(section | section/title | section/title | section/title/text()) ) section/title/text()) ) } }

32. SDPL 2002Notes 9: XQuery32 Querying relational data n Lots of data is stored in relational databases n should be able to access also this data n Example: suppliers and parts –Table S: supplier numbers (sno) and names (sname) –Table P: part numbers (pno) and descriptions (descrip) –Table SP: relationships between suppliers and the parts they supply, including the price (price) of each part from each supplier

33. SDPL 2002Notes 9: XQuery33 A possible XML form for relations * * *

34. SDPL 2002Notes 9: XQuery34 SQL vs. XQuery n SQL: n XQuery: SELECT pno FROM p WHERE descrip LIKE ’Gear%’ ORDER BY pno; for $p in document(”p.xml”)//p_tuple where starts-with($p/descrip, ”Gear”) return $p/pno sortby(.)

35. SDPL 2002Notes 9: XQuery35 Grouping n Many relational queries involve forming data into groups and applying some aggregation function such as count or avg to each group n in SQL: GROUP BY and HAVING clauses n Example: Find the part number and average price for parts that have at least 3 suppliers

36. SDPL 2002Notes 9: XQuery36 Grouping: SQL SELECT pno, avg(price) AS avgprice FROM sp GROUP BY pno HAVING count(*) >= 3 ORDER BY pno;

37. SDPL 2002Notes 9: XQuery37 Grouping: XQuery for $pn in distinct-values( document(”sp.xml”)//pno) let $sp := document(”sp.xml”)//sp_tuple[ pno = $pn] where count($sp) >= 3 return { { $pn, $pn, {avg($sp/price)} {avg($sp/price)} } } sortby(pno)

38. SDPL 2002Notes 9: XQuery38 Joins n Example: Return a ”flat” list of supplier names and their part descriptions, in alphabetic order for $sp in document(”sp.xml”)//sp_tuple, $p in document(”p.xml”)//p_tuple[ $p in document(”p.xml”)//p_tuple[ pno = $sp/pno], $s in document(”s.xml”)//s_tuple[ sno = $sp/sno] $s in document(”s.xml”)//s_tuple[ sno = $sp/sno] return { $s/sname, $s/sname, $p/descrip $p/descrip } sortby (sname, descrip) } sortby (sname, descrip)

39. SDPL 2002Notes 9: XQuery39 XQuery vs. XSLT n Could we express XQuery queries with XSLT? –In principle yes, always, (but could be tedious) n Ex: Partial XSLT simulation of FLWR expressions XQuery : for $x in Expr … can be expressed in XSLT as : can be expressed in XSLT as : … …

40. SDPL 2002Notes 9: XQuery40 XQuery vs. XSLT XQuery : let $y := Expr … corresponds directly to : corresponds directly to : and where Condition … can be expressed as : … …

41. SDPL 2002Notes 9: XQuery41 XQuery vs. XSLT XQuery : return ElemConstructor can be simulated with a corresponding XSLT template: static fragments as such static fragments as such enclosed expressions in element content, e.g. {$s/sname} become enclosed expressions in element content, e.g. {$s/sname} become

42. SDPL 2002Notes 9: XQuery42 XQuery vs. XSLT: Example for $b in document(”bib.xml”)//book where $b/publisher = ”MK” and $b/year > 1998 return return {$b/title} {$b/title} XQuery: 1998”> 1998”> 

43. SDPL 2002Notes 9: XQuery43 XSLT for XQuery FLWR Expressions n NB: The sketched simulation is not complete: –Only two things can be done in XSLT for result tree fragments produced by templates: »insertion in result tree, and conversion to a string (with xsl:value-of ) –Not possible to apply further operations to results (like, e.g., sorting in XQuery)

44. SDPL 2002Notes 9: XQuery44 Summary n XQuery –W3C working draft of an XML query language –vendor support?? »http://www.w3.org/XML/Query mentions about 20 prototype implementations or products that support (or will support) some version of XQuery –future?? –promising confluence of document and database research