Nick McKeown CS244 Lecture 17 Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications [Stoica et al 2001]

About the author Professor Ion Stoica, UC Berkeley 2

Hashing Q: How does a single-node hash table work? Q: When might we use one? 3

Hashing Single-node hash table: –key = hash(name) –put(key, value) –get(key)  value 4 hash(name) name “cat_photo.jpg” key 0x2bd5f802bddeb12 Value = Store

address 0xeb12 Hashing In a memory with n locations 5 hash(name) name “cat_photo.jpg” key 0x2bd5f802bddeb12 Value = Store n mod n

address 0xeb12 mod n Hashing In a node with m memories each with n locations 6 hash(name) name “cat_photo.jpg” key 0x2bd5f802bddeb12 Store 1 n Value = Store m n mod m store

Scaling hashing Q: If I need to distribute the values over m computers (“nodes”), why don’t I just make m and n large enough to hold m x n values? Q: How would it work with m nodes each with its own name or IP address? Q: When would it not work? 7

Why people care Motivated by Napster, Gnutella, Freenet Four simultaneous papers –Chord (Stoica et al. >12,000 citations ) –CAN (Ratnasamy et al.) –PASTRY (Rowstron et al.) –Tapestry (Zhao et al.)

Distributed Hash Tables Q: Why use this interface? put(index, value) get(index)  value Q: What are its shortcomings? Q: What are the potential uses?

Proposed Uses for DHTs 1. Global file systems [OceanStore, CFS, PAST, Pastiche, UsenetDHT] 2. Naming services [Chord-DNS, Twine, SFR] 3. DB query processing [PIER, Wisc] 4. Internet-scale data structures [PHT, Cone, SkipGraphs] 5. Communication services [i3, MCAN, Bayeux, VRR] 6. Event notification [Scribe, Herald] 7. File sharing [OverNet ]

Background: DHTs Q: What properties would we like a DHT to have? 1.Decentralized: no central authority 2.Scalable: low network traffic overhead 3.Fast: find items quickly 4.Fault-tolerant & dynamic: nodes fail, new nodes join 5.General-purpose: flexible naming 6.Secure

The Lookup Problem N1N1 N2N2 N4N4 N3N3 N5N5 put(key,value) value  get(key) ?

Napster N1N1 N2N2 N4N4 N3N3 N5N5 DB put(key,value) SetLoc(key,N 4 ) value  get(key)

Gnutella N1N1 N2N2 N4N4 N3N3 N5N5 IP  search(key)

Chord Q: What is Chord’s contribution?

Chord goals “The Chord project explored how to build scalable, robust distributed systems using peer-to-peer ideas. The basis for much of our work is the Chord distributed hash lookup primitive. Chord is completely decentralized and symmetric, and can find data using only log(N) messages, where N is the number of nodes in the system. Chord's lookup mechanism is provably robust in the face of frequent node failures and re-joins.” 16

Consistent Hashing [Karger ‘97] 17 N1 N32 N90 N105 Circular 7-bit ID space K5 K20 K80 A key is stored at its successor: the node with next higher ID Q: What are the consequences of how keys are distributed?

Basic Lookup 18 N1 N32 N90 N105 K80 “Where is key 80?” “N90 has key 80”

Basic Lookup Lookup(my-id, key-id) n = my successor if my-id < n < key-id call Lookup(key-id) on node n else return my successor //done (Correctness depends only on successors) 19 Q: How does Chord accelerate the lookup?

Lookup with Fingers 20 Finger i points to successor of n + 2 i (Fingers accelerate queries without affecting correctness)

Lookup with Finger Tables Lookup(my-id, key-id) look in local finger table for highest node n s.t. my-id < n < key-id if n exists call Lookup(key-id) on node n else return my successor // done Q: Fingers accelerate lookups. Do fingers affect correctness of the key? 21

Consistency during “joins” 22 stabilize(): - run periodically by each node. - fixes predecessor pointers.

Consistency Key consistency: All nodes agree on who holds a particular key Data consistency: All nodes agree on the value associated with a key Reachability consistency: All nodes are connected in the ring and can be reached in a lookup. (Chord aims to provide eventual reachability consistency, upon which key and data consistency are built.) 23

Controversy over correctness Using Lightweight Modeling To Understand Chord – P. Zave, ACM CCR April 2012 Claims eventual reachability not guaranteed. Counter-examples suggest simultaneous joins can lead to misordering in the ring. 24

Background DHTs Chord’s Interface key = hash(data) lookup(key)  IP address send-RPC(IP address, PUT, key, value) send-RPC(IP address, GET, key) -> value

Background DHTs Application DHT Lookup Service Node Put(key, value) Get(key) data Lookup(key) Node IP address

Chord Analysis

Chord Discussion OpenDHT: Public DHT service Primary complaint: Too Slow! 99 th latency percentile measured in seconds Median latency sometimes over ½ second Q: What’s going on here? Rhea et al. FIXING the Embarrassing Slowness of OpenDHT on PlanetLab

DHT Performance Problems Few arbitrarily slow nodes cause the problem Disk reads can take tens of seconds Computations can be very slow on overloaded nodes Inter-node latencies can be high Q: What is the paradox here? Q: How can it be fixed?