Fault-tolerant Routing in Peer-to-Peer Systems James Aspnes Zoë Diamadi Gauri Shah Yale University PODC 2002

P2P network Bunch of peers. Store resources identified by keys. Peers subject to crash failures. Goal: locate resources ‘’efficiently’’. Resources Peers Key

Properties of ideal network Data availability Decentralization Fault-tolerance Scalability Load balancing Maintaining network Dynamic node addition/deletion Self-stabilization Efficient searching Incorporating geography Incorporating locality

Early P2P systems Napster ? x x ? x Central server bottleneck Gnutella Inefficient flooding

Tapestry [JKZ’01] Uses Plaxton’s Algorithm: Correct one digit at a time to reach target. Pastry [DR’01] is also similar Node xyz links to *XX, x*X and xy* [* = all digits, X = any digit]

CAN [RFHKS’01] (0,0) (1,0) (0,1)(1,1) Partition d-dimensional co-ordinate space into zones. Nodes own zones and keys hashed to them. Greedy routing: forward to neighbor closest to target. d=2 zone

Chord [SMKKB‘01] Nodes and resources mapped to identifier circle. Routing table: successor nodes at distances Greedy routing: forward to node in routing table closest to target successors identifier circle (n=8)

Common underlying structure Underlying metric space. Nodes embedded in metric space. Location determined by key. Hashing to balance load. Greedy routing. O(log n) space at each node. O(log n) routing time.

Unifying approach v2 HASH v1 v4 v3 v1 v2 v3 v4 Keys Nodes Actual Route Physical Link Virtual Link Virtual Route PHYSICAL NETWORK VIRTUAL OVERLAY NETWORK

Link Distribution Each node independently selects k long-hop links as per some distribution. Links chosen as per Nodes x-d 1 x x-d 2

Abstract model Simple metric space: 1D line. Hash(key) = Metric space location. Short-hop links: immediate neighbors. Long-hops links:inverse-distance distribution. Pr[edge(u,v)] = 1/d(u,v) / Greedy Routing: forward message to neighbor closest to target in metric space. 1/d(u,v’)

What do we care about? Do we get similar upper bounds on routing time with failures? Is it possible to design a link distribution that beats the O(log 2 n) bound for routing given by 1/d distribution? Can we dynamically construct such a network?

Greedy routing with failures Analyze message delivery in phases [Kleinberg ‘99]. Target t Phase 0 Phase 1 Phase 2 Message at node n in phase i: 2 i d(n, t) < 2 i+1 At most (log n + 1) such phases.

[1..log n] long-hop links Suppose each node has k long-hop links. Average time spent in each phase: ((log n)/k). With O(log n) such phases: Total time: O((log 2 n)/k). With failures: Suppose each node/link fails with prob (1-p). Average time spent in each phase: ((log n)/pk). Total time: O((log 2 n)/pk)

Simulation results n= nodes log n=17 links What happens with > log n links?

What do we care about? Do we get similar upper bounds on routing time with failures? Is it possible to design a link distribution that beats the O(log 2 n) bound for routing given by 1/d distribution? Lower bound on routing time as a function of number of links per node. Can we dynamically construct such a network?

Intuition for lower bound [KUW’88] Time needed for a non-increasing real-valued Markov chain X 0, X 1, X 2 …. to drop to 1 bounded by: where = E[X t –X t+1 : X t = z] is a non-decreasing function of z.

Upper bound on time SFONYC z gives lower bound on average crossing speed. ( is non-decreasing so ) x gives upper bound on time. Starting from x, average speed at z =.

Lower bound on time SFONYC z x gives upper bound on average crossing speed. m z = sup This may give too large an estimate, so condition against high bursts of speed. gives lower bound on time.

Tool for lower bound Non-increasing Markov chain: X 0, X 1, X 2 ….., state space S. E[ Time to reach 0 ] T(X 0 )/[ T(X 0 ) + (1- )] Pr[X t – X t+1 U : X t = x] Few long jumps Time from x [no long jumps] Upper bound on speed [no long jumps] E[X t – X t+1 : X t =x, X t – X t+1 < U] m z = sup { : x S, x [z, z+U) }

Applying tool to routing Cannot bound progress of single node with an arbitrary distribution! So use an aggregate chain S t of nodes for collective behavior of nodes in some range. Track ln(|S t |) for recurrence relation. StSt d3d3 d2d2 d1d1 Nodes Links 0 S t+1 0

Lower bounds Random graph G. Node x has k independent links on average. x links to (x-1) and (x+1). Expected time to reach 0 from a Point chosen uniformly from 1..n: (ln 2 n) worse that O(ln n) for a tree: cost of assuming symmetry between nodes. 1-sided routing: 2-sided routing: * link ignored s s link ignored * Probability of choosing links symmetric about 0 and unimodal.

What do we care about? Do we get similar upper bounds on routing time with failures? Is it possible to design a link distribution that beats the O(log 2 n) bound for routing given by 1/d distribution? Can we dynamically construct such a network?

Heuristic for construction New node chooses neighbors using inverse distance distribution. Links to live nodes closest to chosen ones. Selects older nodes to point to it. new node ideal link adjusted link absent node new link initial link older node y x

Open problems Design a self-stabilization mechanism. [Aspnes, Shah ’02] submitted to SODA Does lower bound generalize to multidimensional metric spaces? Analyze security properties such as anonymity and byzantine failures. ? Does backtracking give provably good routing bound?