1. 17 th International World Wide Web Conference 2008 Beijing, China XML Data Dissemination using Automata on top of Structured Overlay Networks Iris Miliaraki Zoi Kaoudi Manolis Koubarakis Department of Informatics and Telecommunications National and Kapodistrian University of Athens

2. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 2 Outline XML Dissemination scenario Problems Background: DHTs Our approach Experiments Future work

3. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 3 XPath/XQuery? XML Dissemination system XML Dissemination system XML Dissemination scenario XPath/XQuery? XML Subscriber Subscriber Publisher Publisher Publisher News monitoring Publication monitoring YFilter XTrie FiST Index-Filter CentralizedDistributed ONYX Gong et al. [ICDE05] XPush Parallel/Hierarchical XTrie Snoeren [SOSP 2001]

4. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 4 XML Dissemination: Broker-based architecture Mesh or tree-based overlays XML Subscriber Publisher Publisher XPath/XQuery? ? XML Subscriber Publisher

5. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 5 Problems Load imbalances

6. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 6 XML Dissemination: Broker-based architecture Systems like ONYX and work of Gong et al. [ICDE05] Mesh or tree-based overlays XML Publisher Publisher Publisher XPath/XQuery? XML Subscriber Subscriber XPath/XQuery?

7. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 7 Problems Load imbalances Centralized control Single point of failure and bottleneck

8. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 8 XML Dissemination: Broker-based architecture Systems like ONYX and work of Gong et al. [ICDE05] Mesh or tree-based overlays XML Publisher Publisher Publisher Subscriber XPath/XQuery? ? XML Subscriber

9. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 9 Problems Load imbalances Centralized control Single point of failure and bottleneck Scalability (size of routing tables)

10. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 10 XML Dissemination: Broker-based architecture Systems like ONYX and work of Gong et al. [ICDE05] Mesh or tree-based overlays XML Publisher Publisher Publisher Subscriber XPath/XQuery? ? XML Subscriber

11. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 11 Background: DHTs Structured overlay networks Solve the item location problem in a distributed and dynamic network of nodes (in O(log N) hops): Let x be some data item. Find x! Distributed version of hash table data structure id=Hash(K) Main operations: Put: given a key (for a data item), map the key onto a node. Get: Find the location of a data item with a given a key. Successor peer responsible peer

12. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 12 XML Dissemination revisited: Structured overlay network architecture XML Subscriber Subscriber Publisher Publisher Publisher XPath/XQuery? ?

13. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 13 Problems revisited Load imbalances Centralized control Single point of failure and bottleneck Scalability (size of routing tables)

14. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 14 Automata-based approaches XFilter and YFilter, ONYX, XTrie, IndexFilter, FiST etc. Main idea Construct an automaton from a set of XPath/Xquery queries Use it as a matching engine against the XML documents

15. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 15 Q1: /dblp/phdthesis/year = 2008 YFilter – NFA Construction 3 year Q1 0 dblp phdthesis 1 2

16. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 16 Q1: /dblp/phdthesis/year = 2008 Q2: /dblp/proceedings/school = Univ. of Athens YFilter – NFA Construction 3 year Q1 0 dblp phdthesis 1 2 5 school Q2 proceedings 4

17. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 17 Q1: /dblp/phdthesis/year = 2008 Q2: /dblp/proceedings/school = Univ. of Athens Q3: /dblp/proceedings/title = XML Dissemination YFilter – NFA Construction 3 year Q1 0 dblp phdthesis 1 2 title Q3 6 5 school Q2 proceedings 4

18. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 18 8 author Q4 Q1: /dblp/phdthesis/year = 2008 Q2: /dblp/proceedings/school = Univ. of Athens Q3: /dblp/proceedings/title = XML Dissemination Q4: /dblp/*/author = John Doe YFilter – NFA Construction 3 year Q1 0 dblp phdthesis 1 2 * 7 title Q3 6 5 school Q2 proceedings 4

19. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 19 ε 9 * Q5: //*/cite = [12743] 11 cite Q5Q5 10 * YFilter – NFA Construction 3 year Q1 0 dblp phdthesis 1 2 8 author Q4 * 7 title Q3 6 5 school Q2 proceedings 4 Q1: /dblp/phdthesis/year = 2008 Q2: /dblp/proceedings/school = Univ. of Athens Q3: /dblp/proceedings/title = XML Dissemination Q4: /dblp/*/author = John Doe

20. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 20 ε 9 * Q5: //*/cite = [12743] 11 cite Q5Q5 10 * YFilter – NFA Construction 3 year Q1 0 dblp phdthesis 1 2 8 author Q4 * 7 title Q3 6 5 school Q2 proceedings 4 Q1: /dblp/phdthesis/year = 2008 Q2: /dblp/proceedings/school = Univ. of Athens Q3: /dblp/proceedings/title = XML Dissemination Q4: /dblp/*/author = John Doe

21. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 21 Main idea Utilize a distributed version of a state-of-the- art approach YFilter Instead of a centralized NFA Distribute the NFA in the DHT

22. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 22 Distributing the NFA on top of DHT P1 P2 P9 P8 P7 P6 P3 P5 P4 P10 State key01234567891011 Successor peerP3P5P1P2P6P7 P8P10P4P9P10 2 3 0 9 1 11 4 10 7 56

23. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 23 Distributing the NFA on top of DHT P1 P2 P9 P8 P7 P6 P3 P5 P4 P10 State key01234567891011 Successor peerP3P5P1P2P6P7 P8P10P4P9P10 2 3 0 9 1 11 4 10 7 56 1247

24. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 24 Distributing the NFA on top of DHT P1 P2 P9 P8 P7 P6 P3 P5 P4 P10 State key01234567891011 Successor peerP3P5P1P2P6P7 P8P10P4P9P10 2 3 0 9 1 11 4 10 7 56 1247 =0 =1

25. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 25 Distributing the NFA on top of DHT State key01234567891011 Successor peerP3P5P1P2P6P7 P8P10P4P9P10 P1 P2 P9 P8 P7 P6 P3 P5 P4 P10 23 3 01910 9 1247 11 456 1011 78 56

26. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 26 YFilter - NFA Execution Univ. of Athens XML and DHTs Incoming XML document These paths can be executed in parallel! 0 1 9 10 4 7 9 10 5 9 10 6 9 10 Runtime stack dblp proceedings school title * ε * * Start of document End of document

27. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 27 Univ. of Athens XML and DHTs Start of document End of document Distributed NFA execution – Iterative Incoming XML document 0 1 9 10 4 7 9 10 5 9 10 6 9 10 Publisher P1 P2 P9 P8 P7 P6 P3 P5 P4 P10 2 3 0 9 1 11 4 10 76 5 Publisher becomes overloaded!

28. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 28 Univ. of Athens XML and DHTs Distributed NFA execution - Recursive Incoming XML document Publisher P1 P2 P9 P8 P7 P6 P3 P5 P4 P10 2 3 0 9 1 11 4 10 76 5 Start of document End of document 0 1 9 10 0 9 0 10 0 1 9 10 4 7 0 1 4 0 1 7 10 9 0 1 4 5 0 1 4 6 0 1 4 0 1 4

29. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 29 Experimental evaluation Chord simulator 2 different document workloads Aggregated Including DBLP, NITF, ebXML, Auction (XMark) NITF 2 kinds of query sets Random Distinct

30. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 30 Metrics Network traffic total number of messages Latency longest chain of hops Filtering load number of messages received during execution

31. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 31 Iterative vs Recursive

32. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 32 Varying number of queries – Network traffic

33. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 33 Varying number of queries - Latency

34. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 34 Load balancing Virtual peers Originally proposed in Chord Mapping of multiple virtual peers to each real peer Load-shedding Replicate on demand

35. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 35 Load balancing – Filtering load

36. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 36 Conclusions DHT-based protocols overcoming weaknesses of broker-based architectures Utilize a distributed YFilter engine Exploit inherent parallelism of an automaton Experimental evaluation

37. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 37 Future Work Implementation and experimenting on an Internet-scale testbed like PlanetLab More sophisticated methods for predicate evaluation

38. 24 April 2008 17 th International World Wide Web Conference 2008 Beijing, China 38 Thank you for your attention Questions?