1. p2p, Fall 06 1 Querying the Internet with PIER (PIER = Peer-to-peer Information Exchange and Retrieval) VLDB 2003 Ryan Huebsch, Joe Hellerstein, Nick Lanham, Boon Thau Loo, Timothy Roscoe, Scott Shenker, Ion Stoica

2. p2p, Fall 06 2 What is PIER? A query engine that scales up to thousands of participating nodes = relational queries + DHT Built on top of a DHT

3. p2p, Fall 06 3 Motivation, Why? In situ distributed querying (as opposed to warehousing) Network monitoring network intrusion detection: sharing and querying fingerprint information Fingerprints (sequences of port accesses, port numbers, packet contents etc) Where: network servers: mail servers, web servers, remote shells) & applications (mail clients, web browsers) Publish the fingerprint of an attack in PIER for a short time Query it – similar reports, statistics, intrusions from the same domain, etc

4. p2p, Fall 06 4 Motivation, Why? Besides security: network statistics (eg bandwidth utilization, etc) In-network processing – streams Text search Deep web crawling

5. p2p, Fall 06 5 Architecture DHT is divided into 3 modules:  Routing Layer  Storage Manager  Provider Goal is to make each simple and replaceable An instance of each DHT and PIER component runs at each participating node PIER is build on top of a DHT (in the paper, the DHT is CAN with d = 4) The DHT is used both for indexing and storage

6. p2p, Fall 06 6 Architecture Routing Layer Maps a key to the IP address of the node currently responsible for it API lookup(key) -> ippaddr join(landmark) leave() LocationMapChange() Callback used to notify higher levels asynchronously when a set of keywords mapped locally has changed Core Relational Execution Engine ProviderStorage Manager Overlay Routing Catalog Manager Query Optimizer Various User Applications PIER DHT Apps

7. p2p, Fall 06 7 Architecture Storage Manager Temporary storage of DHT-based data Local database at each DHT node a simple in memory storage- system or a disk-based package like Berkeley DB In this paper, a main memory storage manager API store(key, item) retrieve(key) -> item remove(key) Core Relational Execution Engine Provider Storage Manager Overlay Routing Catalog Manager Query Optimizer Various User Applications PIER DHT Apps

8. p2p, Fall 06 8 Architecture Provider What PIER sees What are the data items (relations) handled by PIER? Core Relational Execution Engine Provider Storage Manager Overlay Routing Catalog Manager Query Optimizer Various User Applications PIER DHT Apps

9. p2p, Fall 06 9 Naming Scheme Each object: (namespace, resourceID, instanceID) DHT key: hash on namespace, resourceID  Namespace: group, application the object belongs to In PIER, the Relation Name No need to be predefined, can be created implicitly when the first item is put and destroyed when the last item expires  ResourceID: some semantic meaning In PIER, the value of the primary key for base tuples  InstanceID: an integer randomly assigned by the user application Used by the storage manager to separate items (when they are not stored based on the primary key) What are the data items (relations) handled by PIER?

10. p2p, Fall 06 10 Soft State Each object associated with a lifetime: how long should the DHT store the object To extend it, must use periodical RENEW calls

11. p2p, Fall 06 11 Architecture Provider API get(namespace, resourceID) -> item put(namespace, resourceID, instanceID, item, lifetime) renew(namespace, resourceID, instanceID, item, lifetime) -> bool multicast(namespace, resourceID, item) Contacts all nodes that hold data in a particular namespace lscan(namespace) -> iterator Scans over all data stored locally newData(namespace) -> item Callback to the application to inform it that new data has arrived in a particular namespace

12. p2p, Fall 06 12 Architecture  PIER currently only one primary module: the relational execution engine  Executes a pre-optimized query plan  Traditional DBs use an optimizer + catalog to take SQL and generate the query plan, those are “just” add-ons to PIER

13. p2p, Fall 06 13 Architecture Query plan is a box-and-arrow description of how to connect basic operators together  selection, projection, join, group-by/aggregation, and some DHT specific operators such as rehash No iterator model Instead, operators produced results as quickly as possible and enqueue the data for the next operator

14. p2p, Fall 06 14 Joins: The Core of Query Processing R Join S, relations R and S stored in separate namespaces N R and N S How: –Get tuples that have the same value for a particular attribute(s) (the join attribute(s)) to the same site, then append tuples together Why Joins? A relational join can be used to calculate: –The intersection of two sets –Correlate information –Find matching data Algorithms come from existing database literature, minor adaptations to use DHT.

15. p2p, Fall 06 15 Symmetric Hash Join (SHJ) Algorithm for each site –(Scan – Retrieve local data) Use two lscan calls to retrieve all data stored locally from the source tables –(Rehash based on the join attribute) put a copy of each eligible tuple with the hash key based on the value of the join attribute (new unique namespace N Q ) –(Listen) use newData and get to N Q to see the rehashed tuples –(Compute) Run standard one-site join algorithm on the tuples as they arrive Scan/Rehash steps must be run on all sites that store source data Listen/Compute steps can be run on fewer nodes by choosing the hash key differently

16. p2p, Fall 06 16 Fetch Matches (FM) Algorithm for each site –(Scan) Use lscan to retrieve all data from ONE table N R –(Get) Based on the value for the join attribute, for each R tuple issue a get for the possible matching tuples from the S table Big picture: –SHJ is put based –FM is get based When one of the tables, say S is already hashed on the join attribute

17. p2p, Fall 06 17 Joins: Additional Strategies Symmetric Semi-Join –Locally at each site, run a SHJ on the source data projected to only have the hash key and join attributes. –Use the results of this mini-join as source for two FM joins to retrieve the other attributes for tuples that are likely to be in the answer set Bloom Filters –Create Bloom filters for each local R and S data and publish them at a temporary DHT, OR all local R and multicast to nodes hosting S – use lscan but rehash only those that match Big Picture: –Tradeoff bandwidth (extra rehashing) for latency (time to exchange filters)

18. p2p, Fall 06 18 Naïve Group-By/Aggregation A group-by/aggregation can be used to calculate: –Split data into groups based on value –Max, Min, Sum, Count, etc. Goal: –Get tuples that have the same value for a particular attribute(s) (group-by attribute(s)) at the same site, then summarize data (aggregation).

19. p2p, Fall 06 19 Naïve Group-By/Aggregation At each site –(Scan) lscan the source table Determine group tuple belongs in Add tuple’s data to that group’s partial summary –(Rehash) for each group represented at the site, rehash the summary tuple with hash key based on group-by attribute –(Combine) use newData to get partial summaries, combine and produce final result after specified time, number of partial results, or rate of input Hierarchical Aggregation: Can add multiple layers of rehash/combine to reduce fan-in. –Subdivide groups in subgroups by randomly appending a number to the group’s key

20. p2p, Fall 06 20 Naïve Group-By/Aggregation Sources Root Application Overlay Sources Root … Each message may take multiple hops Each level fewer nodes participate

21. p2p, Fall 06 21 Codebase Approximately 17,600 lines of NCSS Java Code Same code (overlay components/pier) run on the simulator or over a real network without changes Runs simple simulations with up to 10k nodes –Limiting factor: 2GB addressable memory for the JVM (in Linux) Runs on Millennium and Planet Lab up to 64 nodes –Limiting factor: Available/working nodes & setup time Code: –Basic implementations of Chord and CAN –Selection, projection, joins (4 methods), and naïve aggregation. –Non-continuous queries

22. p2p, Fall 06 22 Seems to scale Simulations of 1 SHJ Join Warehousing Full Parallelization

23. p2p, Fall 06 23 Some real-world results 1 SHJ Join on Millennium Cluster

24. p2p, Fall 06 24 Other applications Recursive queries on network graphs set of nodes reachable within k hops of each node Hierarchical aggregation Range Predicates locality-preserving hash functions