Tapestry: Finding Nearby Objects in Peer-to-Peer Networks Joint with: Ling Huang Anthony Joseph Robert Krauthgamer John Kubiatowicz Satish Rao Sean Rhea Jeremy Stribling Ben Zhao

Object Location

Behind the Cloud

Why nearby? (DHT vs. DOLR) Nearby= low stretch, ratio of distance traveled to find object to distance to closest copy of object Objects are services, so distance isn’t one-time cost (see COMPASS) (smart) publishers put objects at chosen locations in network –Bob Miller places retreat schedule at node in Berkeley Wildly popular objects

Well-Placed Objects

Popular Objects

Two solutions One directory –One participant stores a lot, but average is low –Stretch is high Everyone has a directory –Low stretch! –Everyone has lots of data –Changes are very expensive

Distribute directory Each peer is responsible for part of directory Q: Who is responsible for my object? A: Hash peers into same namespace as objects. Q: How do I get there? A: Peer-to-peer networks route to names.  Must efficiently self-organize into network

Efficiency is… With respect to network Few neighbors Short routes (few hops & low stretch) Easy routing (No routing tables!) Dynamic With respect to directory Not too many entries (4,4) hard easy

Low stretch dynamic peer-to-peer network Tolerate failures in network Adapting to network variation Future work Outline

SystemNeighbor s Motivating Structure Hops CAN, 2001O(r)gridO(rn 1/r ) Chord, 2001O(log n)hypercubeO(log n) Pastry, 2001O(log n)hypercubeO(log n) Tapestry, 2001O(log n)hypercubeO(log n) Distributed Hash Tables These systems give –Guaranteed location –Join and leave algorithms –Load-balanced storage No stretch guarantees

Low Stretch Approaches SystemStretchSpaceBalancedMetric Awerbuch Peleg, 1991 polylog noGeneral PRR, 1997O(1)O(log n)yesSpecial Thorup-ZwickO(k 2 )O(kn 1/k )yesGeneral RRVV, 2001polylog yesGeneral Not dynamic Tapestry is first dynamic low-stretch scheme

PRR/Tapestry City State Country

PRR/Tapestry Two object types: red and blue, so two trees Level 1 Level 2 Level 3

Neighbor Table For “5471” (Octal) Routing Levels xxx 2xxx 0xxx 3xxx 4xxx 5xxx 6xxx 7xxx 50xx 51xx 52xx 53xx 54xx 55xx 56xx 57xx 540x 541x 542x 543x 544x 545x 546x 547x Ø 5475 Ø 5477 Balancing Load 1 NodeID NodeID 5471 NodeID 5416 NodeID 5061 NodeID 5432 NodeID 5455 NodeID 5470

Building It Every node is a root! Pefix routing in base b  promoting at rate 1/b b=16  match one hex digit or four bits at a time Space: O(b log b n) –Every node at every level stores b parents We want the links to be of geometrically increasing length

Big Challenge: Joining Nodes Theorem 1 [HKRZ02] When peer A is finished inserting, it knows about all relevant peers that have finished insertion.

Results Correctness O(log n) insert & delete –Concurrent inserts in a lock-free fashion Neighbor-search routine –Required to keep low stretch –All low-stretch schemes do something like this Zhao, Huang, Stribling, Rhea, Joseph & Kubiatowicz (JSAC) –This works! Implemented algorithms –Measured performance

Neighbor Search In growth-restricted networks (with no additional space!): Theorem 2 [HKRZ02] Can find nearest neighbor with high probability with O(log 2 n) messages Theorem 3 [HKMR04] Can find nearest neighbor, and messages is O(log n) with high probability

Algorithm Idea, in pictures new

Low stretch dynamic peer-to-peer network Tolerate failures in network Adapting to network variation Future work Outline

Behind the Cloud Again

Dealing with faults Multiple paths –Castro et. al –One failure along path, path breaks Wide path –Paths faulty at the same place to break Exponential difference in width effect “retrofit” Tapestry to do latter in slightly malicious networks Failed! Still good…

Effective even for small overhead Theorem 4 In growth restricted spaces, can make probability of failed route less than 1/n c for width O(clog n) Hildrum & Kubiatowicz, DISC02

Wide path vs. multiple paths

Low stretch dynamic peer-to-peer network Tolerate failures in network Adapting to Network Variation Future work Outline

Digit size affects performance

Network not homogeneous Previous schemes picked a digit size How do we find a good one? But what if there isn’t one? San Francisco Nebraska Paris

New Result Pick digit size based on local measurements Don’t need to guess Vary digit size depending on location –No, it’s not obvious that this works, but it does! Hildrum, Krauthgamer & Kubiatowicz [SPAA04]: Dynamic, locally optimal low-stretch network

Conclusions and Future Work Conclusion –Low stretch object location is practical System provably good [HKRZ02] System built [ZHSJK] Open Questions –Do we need a DOLR? Object placement schemes? Workload? –Examples where low stretch, load balance, and low storage not possible simultaneously What is tradeoff between degree, stretch, load balance as function of graph? Can we get best possible? Trade off smoothly?

Tapestry People Ling Huang Anthony Joseph John Kubiatowicz Sean Rhea Jeremy Stribling Ben Zhao and…OceanStore group members