Presentation is loading. Please wait.

Presentation is loading. Please wait.

Estimating the Selectivity of XML Path Expressions for Internet Scale Applications Ashraf Aboulnaga Alaa R. Alameldeen Jeffrey F. Naughton Computer Sciences.

Published byJared Charles Modified over 9 years ago

Similar presentations

Presentation on theme: "Estimating the Selectivity of XML Path Expressions for Internet Scale Applications Ashraf Aboulnaga Alaa R. Alameldeen Jeffrey F. Naughton Computer Sciences."— Presentation transcript:

1 Estimating the Selectivity of XML Path Expressions for Internet Scale Applications Ashraf Aboulnaga Alaa R. Alameldeen Jeffrey F. Naughton Computer Sciences Department University of Wisconsin - Madison

2 Motivation XML enables Internet scale applications that query data from many sources Niagara, Xyleme, … Queries over XML data use path expressions Optimizing these queries requires estimating the selectivity of the path expressions Focus of this talk: Building statistics for XML data and using them for estimating the selectivity of simple path expressions

3 What is XML? Pygmalion Bernard Shaw David Copperfield Charles Dickens

4 Querying XML FOR $n_auth IN document("*")//novel/author $p_auth IN document("*")//play/author WHERE $n_auth/text() = $p_auth/text() RETURN $n_auth Optimizing this query requires estimating the selectivity of the path expressions This requires information about the structure of the XML data

5 Goal of this Work Build database statistics that capture the structure of XML data Ensure that the statistics fit in a small amount of memory For efficient query optimization Important for Internet scale applications Use the statistics to estimate the selectivity of simple XML path expressions //t1/t2/…/tn

6 Outline of Presentation Introduction Path Trees Markov Tables Performance Evaluation Conclusions

7 Path Trees A 1 C 1 B 2 D 1 D 1 E 3

8 Summarizing Path Trees Path trees contain all the information needed for selectivity estimation Problem: May not fit in available memory Small available memory Internet scale Remove low frequency nodes Removed nodes replaced with *-nodes Tag name: * meaning "any tag" Frequency: Average frequency of replaced nodes Sibling-*, Level-*, Global-*, No-*

9 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

10 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

11 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

12 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

13 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 * f=6 n=2 *-nodes represent deleted sibling nodes Memory saved by coalescing nodes

14 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 * f=6 n=2

15 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 * f=6 n=2

16 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 * f=6 n=2

17 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 K 11 * f=6 n=2 * f=12 n=2

18 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 * K 11 * f=6 n=2 f=12 n=2

19 Sibling-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 * K 11 * f=6 n=2 f=12 n=2

20 Sibling-* Summarization A 1 C 9 B 13 F 15 K 12 * K 11 * f=6 n=2 f=12 n=2 * f=16 n=2

21 Sibling-* Summarization A 1 C 9 B 13 * F 15 * K* f=6 n=2 f=12 n=2 f=16 n=2 f=23 n=2

22 Sibling-* Summarization A 1 C 9 B 13 * F 15 * K* f=23 n=2 68 3

23 Original Path Tree A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

24 Sibling-* Summarization A 1 C 9 B 13 * F 15 * K* f=23 n=2 68 3 Try to retain as much information as possible about the deleted nodes

25 Level-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

26 Level-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

27 Level-* Summarization A 1 C 9 B 13 G 10 F 15 K 12 K 11 * 6 * 3 Less information about deleted nodes than sibling-* Deletes fewer nodes than sibling-*

28 Global-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

29 Global-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

30 Global-* Summarization C 9 B 13 G 10 F 15 H 6 K 12 D 7 K 11 * 3

31 No-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

32 No-* Summarization A 1 C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 J 4 I 2

33 No-* Summarization C 9 B 13 G 10 F 15 H 6 K 12 E 5 D 7 K 11 Memory savings similar to global-* Conservative assumption about deleted nodes

34 Outline Introduction Path Trees Markov Tables Performance Evaluation Conclusions

35 Markov Tables A table of all distinct paths of length up to m and their frequencies For paths of length greater than m, combine paths from the Markov table Example: Uses "short memory" or "Markov" property f(B/C/D) f(B/C) f(A/B/C/D) = f(A/B/C)

36 Markov Tables PathFreqPathFreq A1AC6 B11AD4 C15BC9 D19BD7 AB11CD8 A 1 D 4 C 6 B 11 D 7 C 9 D 8

37 Summarizing Markov Tables Exact selectivities for paths of length up to m Approximate selectivities for paths longer than m Problem: May not fit in available memory Remove low frequency paths Discard removed paths of length > 2 Replace removed paths of length 1 or 2 with *-paths Suffix-*, Global-*, No-*

38 Suffix-* Summarization PathFreqPathFreq A1AC6 B11AD4 C15BC9 D19BD7 AB11CD8

39 Suffix-* Summarization PathFreqPathFreq A1AC6 B11AD4 C15BC9 D19BD7 AB11CD8 *0**0

40 Suffix-* Summarization PathFreqPathFreq A1AC6 B11AD4 C15BC9 D19BD7 AB11CD8 *0**0

41 Suffix-* Summarization PathFreqPathFreq AC6 B11AD4 C15BC9 D19BD7 AB11CD8 *f=1,n=1**0

42 Suffix-* Summarization PathFreqPathFreq AC6 B11AD4 C15BC9 D19BD7 AB11CD8 *f=1,n=1**0 S D = { } Set of deleted paths of length 2

43 Suffix-* Summarization PathFreqPathFreq AC6 B11 C15BC9 D19BD7 AB11CD8 *f=1,n=1**0 S D = { (AD,4) }

44 Suffix-* Summarization PathFreqPathFreq AC6 B11 C15BC9 D19BD7 AB11CD8 *f=1,n=1**0 S D = { (AD,4) }

45 Suffix-* Summarization PathFreqPathFreq AC6 B11 C15BC9 D19BD7 AB11CD8 *f=1,n=1**0 S D = { (AD,4) }

46 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15BC9 D19BD7 AB11CD8 *f=1,n=1**0 S D = { }

47 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15BC9 D19BD7 AB11CD8 *f=1,n=1**0 S D = { }

48 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15BC9 D19 AB11CD8 *f=1,n=1**0 S D = { (BD,7) }

49 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15BC9 D19 AB11CD8 *f=1,n=1**0 S D = { (BD,7) }

50 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15BC9 D19 AB11 *f=1,n=1**0 S D = { (BD,7), (CD,8) }

51 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15BC9 D19 AB11 *f=1,n=1**0 S D = { (BD,7), (CD,8) }

52 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15BC9 D19 AB11 *f=1,n=1**0 S D = { (BD,7), (CD,8) }

53 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15B*f=16,n=2 D19 AB11 *f=1,n=1**0 S D = { (CD,8) }

54 Suffix-* Summarization PathFreqPathFreq A*f=10,n=2 B11 C15B*f=16,n=2 D19 AB11 *f=1,n=1**0 S D = { (CD,8) }

55 Suffix-* Summarization PathFreqPathFreq B11 C15B*f=16,n=2 D19 AB11 *f=1,n=1**f=10,n=2 S D = { (CD,8) }

56 Suffix-* Summarization PathFreqPathFreq B11 C15B*8 D19 AB11 *1**6 S D = { }

57 Global-*, No-* Summarization Global-* Two *-paths, * and ** Deletes fewer paths than suffix-* to summarize the Markov table No-* No *-paths Conservatively assumes that paths not in the Markov table do not exist in the data

58 Outline Introduction Path Trees Markov Tables Performance Evaluation Conclusions

59 Data Sets for Experiments Synthetic data set 100,000 XML elements Path tree: 3197 nodes, 6 levels, 38 KB Element frequencies: Zipfian (z=1) DBLP data set 1,399,765 XML elements Path tree: 5883 nodes, 6 levels, 69 KB

60 Query Workloads 1,000 paths of length between 1 and 4 Random paths All query paths exist in the data Random tags Most query paths of length 2 or more do not exist in the data Available memory between 5 and 50 KB

61 Best Summarization Methods Path trees Query paths in data: Global-* Query paths not in data: No-* Markov tables m = 2 is best Query paths in data: Suffix-* Query paths not in data: No-*

62 Path Trees vs. Markov Tables When to use path trees and when to use Markov tables? Also compared against Pruned Suffix Trees (PSTs) [Chen et al, ICDE 2001] Can handle branching path expressions Can handle conditions on element values

63 Synthetic Data – Random Paths

64 Synthetic Data – Random Tags

65 DBLP Data – Random Paths

66 DBLP Data – Random Tags

67 When are Markov Tables Better? DBLP Repeated sub-structures effectively captured by Markov tables … … …

68 Conclusions Novel statistics for estimating the selectivity of XML path expressions Scale to "all the XML data on the Internet" More accurate than best previously known alternative Repeated sub-structures: Markov tables No repeated sub-structures: Path trees Query paths exist in the data: Global-*, Suffix-* Query paths do not exist in the data: No-* To appear in VLDB 2001

Download ppt "Estimating the Selectivity of XML Path Expressions for Internet Scale Applications Ashraf Aboulnaga Alaa R. Alameldeen Jeffrey F. Naughton Computer Sciences."

Similar presentations

Online Mining of Frequent Query Trees over XML Data Streams Hua-Fu Li*, Man-Kwan Shan and Suh-Yin Lee Department of Computer Science.

Online Mining of Frequent Query Trees over XML Data Streams Hua-Fu Li*, Man-Kwan Shan and Suh-Yin Lee Department of Computer Science.
Recap: Mining association rules from large datasets

Recap: Mining association rules from large datasets
Extending Q-Grams to Estimate Selectivity of String Matching with Low Edit Distance [1] Pirooz Chubak May 22, 2008.

Extending Q-Grams to Estimate Selectivity of String Matching with Low Edit Distance [1] Pirooz Chubak May 22, 2008.
A General Algorithm for Subtree Similarity-Search The Hebrew University of Jerusalem ICDE 2014, Chicago, USA Sara Cohen, Nerya Or 1.

A General Algorithm for Subtree Similarity-Search The Hebrew University of Jerusalem ICDE 2014, Chicago, USA Sara Cohen, Nerya Or 1.
Informed Search Methods How can we improve searching strategy by using intelligence? Map example: Heuristic: Expand those nodes closest in “as the crow.

Informed Search Methods How can we improve searching strategy by using intelligence? Map example: Heuristic: Expand those nodes closest in “as the crow.
Query Optimization of Frequent Itemset Mining on Multiple Databases Mining on Multiple Databases David Fuhry Department of Computer Science Kent State.

Query Optimization of Frequent Itemset Mining on Multiple Databases Mining on Multiple Databases David Fuhry Department of Computer Science Kent State.
Frequent Closed Pattern Search By Row and Feature Enumeration

Frequent Closed Pattern Search By Row and Feature Enumeration
CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 15 Nov, 1, 2011 Slide credit: C. Conati, S.

CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 15 Nov, 1, 2011 Slide credit: C. Conati, S.
Fast Algorithms For Hierarchical Range Histogram Constructions

Fast Algorithms For Hierarchical Range Histogram Constructions
Lecture 10 Query Optimization II Automatic Database Design.

Lecture 10 Query Optimization II Automatic Database Design.
Simplifying CFGs There are several ways in which context-free grammars can be simplified. One natural way is to eliminate useless symbols those that cannot.

Simplifying CFGs There are several ways in which context-free grammars can be simplified. One natural way is to eliminate useless symbols those that cannot.
Relational Databases for Querying XML Documents: Limitations & Opportunities VLDB`99 Shanmugasundaram, J., Tufte, K., He, G., Zhang, C., DeWitt, D., Naughton,

Relational Databases for Querying XML Documents: Limitations & Opportunities VLDB`99 Shanmugasundaram, J., Tufte, K., He, G., Zhang, C., DeWitt, D., Naughton,
Database Management Systems 3ed, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 14, Part B.

Database Management Systems 3ed, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 14, Part B.
Database Management Systems, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 12, Part B.

Database Management Systems, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 12, Part B.
FP (FREQUENT PATTERN)-GROWTH ALGORITHM ERTAN LJAJIĆ, 3392/2013 Elektrotehnički fakultet Univerziteta u Beogradu.

FP (FREQUENT PATTERN)-GROWTH ALGORITHM ERTAN LJAJIĆ, 3392/2013 Elektrotehnički fakultet Univerziteta u Beogradu.
Artificial Intelligence Lecture

Artificial Intelligence Lecture
In Search of a More Probable Parse: Experiments with DOP* and the Penn Chinese Treebank Aaron Meyers Linguistics 490 Winter 2009.

In Search of a More Probable Parse: Experiments with DOP* and the Penn Chinese Treebank Aaron Meyers Linguistics 490 Winter 2009.
An Approach to Evaluate Data Trustworthiness Based on Data Provenance Department of Computer Science Purdue University.

An Approach to Evaluate Data Trustworthiness Based on Data Provenance Department of Computer Science Purdue University.
A Generic Framework for Handling Uncertain Data with Local Correlations Xiang Lian and Lei Chen Department of Computer Science and Engineering The Hong.

A Generic Framework for Handling Uncertain Data with Local Correlations Xiang Lian and Lei Chen Department of Computer Science and Engineering The Hong.
Using Structure Indices for Efficient Approximation of Network Properties Matthew J. Rattigan, Marc Maier, and David Jensen University of Massachusetts.

Using Structure Indices for Efficient Approximation of Network Properties Matthew J. Rattigan, Marc Maier, and David Jensen University of Massachusetts.

Similar presentations

Ads by Google