1 Lecture 13: XQuery XML Publishing, XML Storage Monday, October 28, 2002

2 Organization Project: Next phase, need to form companies Please form group of 3,  Tessa by Thursday Problems, little extra-credit: –One group will have only two people –One billing, one shipping volunteers to do inventory Homework: Good practice for the midterm Try to finish before Monday

3 Organization Midterm Next Monday, 11/3 Missed it ? You will get this score: –MidtermScore = 100 – 1.2(100 – FinalScore) –In other words, you will loose 20% more points than on the final

4 Overview DTDs: elements and attributes XQuery

5 Very Simple DTD <!DOCTYPE company [ ]> <!DOCTYPE company [ ]>

6 Very Simple DTD John B Jim B John B Jim B Example of valid XML document:

7 Content Model Element content: what we can put in an element (aka content model) Content model: –Complex = a regular expression over other elements –Text-only = #PCDATA –Empty = EMPTY –Any = ANY –Mixed content = (#PCDATA | A | B | C)* (i.e. very restrictied)

8 Attributes in DTDs

9 Attributes in DTDs <!ATTLIST person age CDATA #REQUIRED id ID #REQUIRED manager IDREF #REQUIRED manages IDREFS #REQUIRED > <!ATTLIST person age CDATA #REQUIRED id ID #REQUIRED manager IDREF #REQUIRED manages IDREFS #REQUIRED > <person age=“25” id=“p29432” manager=“p48293” manages=“p34982 p423234”> <person age=“25” id=“p29432” manager=“p48293” manages=“p34982 p423234”>

10 Attributes in DTDs Types: CDATA = string ID = key IDREF = foreign key IDREFS = foreign keys separated by space (Monday | Wednesday | Friday) = enumeration NMTOKEN = must be a valid XML name NMTOKENS = multiple valid XML names ENTITY = you don’t want to know this

11 Attributes in DTDs Kind: #REQUIRED #IMPLIED = optional value = default value value #FIXED = the only value allowed

12 Using DTDs Must include in the XML document Either include the entire DTD: – Or include a reference to it: – Or mix the two... (e.g. to override the external definition)

13 FLWR (“Flower”) Expressions FOR... LET... WHERE... RETURN... FOR... LET... WHERE... RETURN...

14 Addison-Wesley Serge Abiteboul Rick Hull Victor Vianu Foundations of Databases 1995 Freeman Jeffrey D. Ullman Principles of Database and Knowledge Base Systems 1998 Addison-Wesley Serge Abiteboul Rick Hull Victor Vianu Foundations of Databases 1995 Freeman Jeffrey D. Ullman Principles of Database and Knowledge Base Systems 1998 Sample Data for Queries (more or less)

15 FOR-WHERE-RETURN Find all book titles published after 1995: FOR $x IN document("bib.xml")/bib/book WHERE $x/year/text() > 1995 RETURN $x/title FOR $x IN document("bib.xml")/bib/book WHERE $x/year/text() > 1995 RETURN $x/title Result: abc def ghi

16 FOR-WHERE-RETURN Equivalently (perhaps more geekish) FOR $x IN document("bib.xml")/bib/book[year/text() > 1995] /title RETURN $x FOR $x IN document("bib.xml")/bib/book[year/text() > 1995] /title RETURN $x And even shorter: document("bib.xml")/bib/book[year/text() > 1995] /title

17 FOR-WHERE-RETURN Find all book titles and the year when they were published: FOR $x IN document("bib.xml")/ bib/book RETURN { $x/title/text() } { $x/year/text() }

18 FOR-WHERE-RETURN Notice the use of “{“ and “}” What is the result without them ? FOR $x IN document("bib.xml")/ bib/book RETURN $x/title/text() $x/year/text()

19 XQuery: Nesting For each author of a book by Morgan Kaufmann, list all books she published: FOR $b IN document(“bib.xml”)/bib, $a IN $b/book[publisher /text()=“Morgan Kaufmann”]/author RETURN { $a, FOR $t IN $b/book[author/text()=$a/text()]/title RETURN $t } FOR $b IN document(“bib.xml”)/bib, $a IN $b/book[publisher /text()=“Morgan Kaufmann”]/author RETURN { $a, FOR $t IN $b/book[author/text()=$a/text()]/title RETURN $t } In the RETURN clause comma concatenates XML fragments

20 XQuery Jones abc def Smith ghi Jones abc def Smith ghi Result:

21 Aggregates Find all books with more than 3 authors: count = a function that counts avg = computes the average sum = computes the sum distinct-values = eliminates duplicates FOR $x IN document("bib.xml")/bib/book WHERE count($x/author)>3 RETURN $x

22 Aggregates Same thing: FOR $x IN document("bib.xml")/bib/book[count(author)>3] RETURN $x

23 Aggregates Print all authors who published more than 3 books – be aware of duplicates ! FOR $b IN document("bib.xml")/bib, $a IN distinct-values($b/book/author/text()) WHERE count($b/book[author/text()=$a)>3 RETURN { $a }

24 XQuery Find books whose price is larger than average: FOR $b in document(“bib.xml”)/bib LET $a:=avg($b/book/price/text()) FOR $x in $b/book WHERE $x/price/text() > $a RETURN $x FOR $b in document(“bib.xml”)/bib LET $a:=avg($b/book/price/text()) FOR $x in $b/book WHERE $x/price/text() > $a RETURN $x

25 FOR-WHERE-RETURN “Flatten” the authors, i.e. return a list of (author, title) pairs FOR $b IN document("bib.xml")/bib/book, $x IN $b/title/text(), $y IN $b/author/text() RETURN { $x } { $y } Result: abc efg abc hkj

26 FOR-WHERE-RETURN For each author, return all titles of her/his books FOR $b IN document("bib.xml")/bib, $x IN $b/book/author/text() RETURN { $x } { FOR $y IN $b/book[author/text()=$x]/title RETURN $y } What about duplicate authors ? Result: efg abc klm....

27 FOR-WHERE-RETURN Same, but eliminate duplicate authors: FOR $b IN document("bib.xml")/bib LET $a := distinct-values($b/book/author/text()) FOR $x IN $a RETURN $x { FOR $y IN $b/book[author/text()=$x]/title RETURN $y }

28 FOR-WHERE-RETURN Same thing: FOR $b IN document("bib.xml")/bib, $x IN distinct-values($b/book/author/text()) RETURN $x { FOR $y IN $b/book[author/text()=$x]/title RETURN $y }

29 FOR-WHERE-RETURN Find book titles by the coauthors of “Database Theory”: FOR $b IN document("bib.xml")/bib, $x IN $b/book[title/text() = “Database Theory”], $y IN $b/book[author/text() = $x/author/text()] RETURN { $y/title/text() } Result: abc def abc ghk Question: Why do we get duplicates ?

30 Distinct-values Same as before, but eliminate duplicates: Result: abc def ghk distinct-values = a function that eliminates duplicates Need to apply to a collection of text values, not of elements – note how query has changed FOR $b IN document("bib.xml")/bib, $x IN $b/book[title/text() = “Database Theory”]/author/text(), $y IN distinct-values($b/book[author/text() = $x] /title/text()) RETURN { $y } FOR $b IN document("bib.xml")/bib, $x IN $b/book[title/text() = “Database Theory”]/author/text(), $y IN distinct-values($b/book[author/text() = $x] /title/text()) RETURN { $y }

31 SQL and XQuery Side-by-side Product(pid, name, maker) Company(cid, name, city) Find all products made in Seattle SELECT x.name FROM Product x, Company y WHERE x.maker=y.cid and y.city=“Seattle” FOR $r in document(“db.xml”)/db, $x in $r/Product/row, $y in $r/Company/row WHERE $x/maker/text()=$y/cid/text() and $y/city/text() = “Seattle” RETURN { $x/name } SQL XQuery FOR $y in /db/Company/row[city/text()=“Seattle”], $x in /db/Product/row[maker/text()=$y/cid/text()] RETURN { $x/name } Cool XQuery

32 ??? …. …....

33 XQuery FOR $x in expr -- binds $x to each value in the list expr LET $x := expr -- binds $x to the entire list expr –Useful for common subexpressions and for aggregations

34 XQuery $b is a collection of elements, not a single element count = a (aggregate) function that returns the number of elms { FOR $p IN distinct-values(//publisher/text()) LET $b := /db/book[publisher/text() = $p] WHERE count($b) > 100 RETURN { $p } } { FOR $p IN distinct-values(//publisher/text()) LET $b := /db/book[publisher/text() = $p] WHERE count($b) > 100 RETURN { $p } } Find all publishers that published more than 100 books:

35 XQuery Summary: FOR-LET-WHERE-RETURN = FLWR FOR/LET Clauses WHERE Clause RETURN Clause List of tuples Instance of Xquery data model

36 FOR v.s. LET FOR Binds node variables  iteration LET Binds collection variables  one value

37 FOR v.s. LET FOR $x IN /bib/book RETURN { $x } FOR $x IN /bib/book RETURN { $x } Returns:... LET $x := /bib/book RETURN { $x } LET $x := /bib/book RETURN { $x } Returns:...

38 Collections in XQuery Ordered and unordered collections –/bib/book/author/text() = an ordered collection: result is in document order –distinct-values(/bib/book/author/text()) = an unordered collection: the output order is implementation dependent LET $a := /bib/book  $a is a collection $b/author  a collection (several authors...) RETURN { $b/author } Returns:...