Looking Up Data in P2P Systems Hari Balakrishnan M.Frans Kaashoek David Karger Robert Morris Ion Stoica

Outline The Lookup Problem A Distributed Hash Table Routing in One Dimension Routing in Multiple Dimensions Summary and Open Questions

The Lookup Problem To find a data item X stored at some dynamic set of nodes in the system.  Problem in Distributed Systems  Critical common problem in P2P systems Solutions can be:  Structured lookups: Maintain central database. Disadvantage: central point of failure

The Lookup Problem Using Hierarchy Disadvantage: failure or removal of the root or a node high in the hierarchy Advantage of Structured lookups: data can be reliably found in the system once it is stored.  Symmetric lookup algorithms  Queries are forwarded from nodes to nodes.  Can be started from any node Recent P2P algorithms: CAN, Chord, Kademlia, Pastry, Tapestry, and Viceroy are both structured and symmetric

A Distributed Hash Table Attractive foundation for a distributed lookup algorithm Data is identified with unique numeric keys, and nodes store keys for each other Implements one operation: lookup(key) Finds a node currently responsible for the given key.

A Distributed Hash Table To publish a file: Converts the name to a numeric key using hash function, then calls lookup(key). Then sends the file to be stored at the node(s) responsible for the key. To read: Obtains its name, converts to key, calls lookup(key), and asks the resulting node for a copy of the file.

A Distributed Hash Table(Cont…) To implement DHTs, lookup algorithms have to address the following issues: Mapping keys to nodes in a load-balanced way Forwarding a lookup for a key to an appropriate node Distance Function Building routing tables adaptively

Routing in One Dimension Chord: Skip-list routing Pastry, Kademlia, Tapestry: Tree-like routing Viceroy:Butterfly  Requires information about only constant other number nodes

Routing in One Dimension Chord: Skiplist-like routing N8 N21 N1 N14 N48 N51 N56 N38 N42 Lookup(54) K54

Routing in One Dimension Both keys and nodes are one-dimensional Successor- the node with the closest succeeding ID Finger Table(routing table): Contains first node succeeding n by at least i.e. new node=successor(n+ ) successor(1)=1 successor(2)=3 … i-th 6 successor(6)=

Routing in One Dimension Pastry: Tree-list like routing Uses a prefix-based forwarding scheme Each node maintains a leaf set L, |L|:2 closest and >n and |L|:2 closest and<n To optimize performance uses Routing table  To find a node whose first some digits is same with n’s If sought key is covered by n’s leaf set, then the query is forwarded to that node, else forwarded to a node from a routing table that has a longer shared prefix than n with the sought key

Routing in Multiple Dimension CAN(Content-Addressable Network) Uses d-dimensional Cartesian coordinate space Node is identified by boundaries of its zone Each node maintains a routing table of neighbors

Routing in Multiple Dimension (0,1)(1,1) (1,0)(0,0) (0,1) (1,1) (1,0)(0,0) (0,0.5,0.5,1) (0,0.25,0.75,0.5) (0.5,0.5,1,1) (0,0,0.5,0.5) (0.75,0,1,0.5) Path of “lookup(0.8,0.9)” Initiated at node (0,0,0.5,0.5) Key=(0.8,0.9) stored at node (0.75,0.75,1,1)

Summary and Open Questions Distance function Operation costs Fault Tolerance and concurrent changes Proximity routing Malicious nodes