Vincent Matossian September 21st 2001 ECE 579 An Overview of Decentralized Discovery mechanisms

Decentralized Discovery mechanisms Centralized indexes and repositories Flooding broadcast of queries Selective forwarding/routing of queries Decentralized hashing index systems Distributed indexes and repositories

Centralized indexes and repositories

Napster 1 Central Napster server (xyz.mp3, ) From Sylvia Ratnasamy Berkeley

Napster 2 Central Napster server xyz.mp3 ? From Sylvia Ratnasamy Berkeley

Napster 3 Central Napster server xyz.mp3 ? From Sylvia Ratnasamy Berkeley

Drawbacks Advantages Single point of failure Scalability Cost increases with popularity Lawsuits Performance Control of accesses

Decentralized Discovery mechanisms Centralized indexes and repositories Flooding broadcast of queries Selective forwarding/routing of queries Decentralized hashing index systems Distributed indexes and repositories

Flooding broadcast

From Sylvia Ratnasamy Berkeley Gnutella step 1

xyz.mp3 ? From Sylvia Ratnasamy Berkeley Gnutella step 2

Gnutella step 3 From Sylvia Ratnasamy Berkeley

xyz.mp3 Gnutella step 4 From Sylvia Ratnasamy Berkeley

Drawbacks Advantages Message broadcasting becomes a problem as popularity increases due to bandwidth requirements Susceptible to malicious attacks Simple Efficient Flexible query interpretation Reliable in small networks

Decentralized Discovery mechanisms Centralized indexes and repositories Flooding broadcast of queries Selective forwarding/routing of queries Decentralized hashing index systems Distributed indexes and repositories

Clip2 Reflector (Gnutella) CANCELLED

FastTrack (KaZaA Morpheus) Nodes become supernodes automatically if they have sufficient bandwidth and processing power.

Drawbacks Advantages Susceptible to malicious activities Too much importance on Super Nodes Each peer must contain additional information used to route or direct queries received. Performance Scalability Fault-Tolerance

Decentralized Discovery mechanisms Centralized indexes and repositories Flooding broadcast of queries Selective forwarding/routing of queries Decentralized hashing index systems Distributed indexes and repositories

Selective forwarding of queries Chord Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, and Hari Balakrishnan; MIT Content-Addressable Networks Sylvia Ratnasamy, Paul Francis, Mark Handley, Richard Karp, Scott Shenker; UC Berkeley Pastry Antony Rowstron (Rice University) and Peter Druschel (Microsoft) Tapestry Ben Y. Zhao, John Kubiatowicz and Anthony D.Joseph UC Berkeley

N4N4 Publisher Client N6N6 N9N9 N7N7 N8N8 N3N3 N2N2 N1N1 Lookup(“title”) Key=“title” Value=MP3 data… Concept From Robert Morris MIT

Chord 1 N32 N90 N105 K80 K20 K5 Circular 7-bit ID space Key 5 Node 105 A key is stored at its successor: node with next higher ID

Chord 2 N32 N90 N105 N60 N10 N120 K80 “Where is key 80?” “N90 has K80”

Content-Addressable Networks - CAN hash(K) = (a,b) (K,V) retrieve (K) insert (K,V) (a,b)

Bootstrap node 1) Discover some node “I” already in CAN new node CAN Node Insertion

I Bootstrap node new node 1) Discover some node “I” already in CAN CAN Node Insertion

2) pick random point in space I (p,q) new node CAN Node Insertion

(p,q) 3) I routes to (p,q), discovers node J I J new node CAN Node Insertion

new J 4) split J’s zone in half… new owns one half CAN Node Insertion

Example: Octal digits, 2 18 namespace,  Neighbor Map For “5712” (Octal) Routing Levels 1234 xxx xxx0 xxx3 xxx4 xxx5 xxx6 xxx7 xx xx22 xx32 xx42 xx52 xx62 xx72 x012 x112 x212 x312 x412 x512 x Plaxton Rajamaran Richa

PASTRY TAPESTRY Based on Plaxton Rajamaran Richa algorithm but have additional support for dynamic node insertion and deletion. Node insertion: Node N requests a new ID and contacts a Gateway G. Neighbor maps tables are updated along each hop. Minor differences in object replications and routing distances calculation.

Comparing Key Metrics Properties –Parameter –Logical Path Length –Neighbor-state –Routing Overhead (RDP) –Messages to insert –Mutability –Load-balancing Tapestry ChordCANPastry Log b N Log 2 N O(d*N 1/d ) O(d) Base b bLog b N O(1)  O(1) O(1) ? App-dep. App-depImmut. ??? Good Log b N Log 2 N NoneDimen dBase b bLog b N+O(b) O(Log 2 2 N)O(d*N 1/d ) Good O(Log b 2 N)O(Log b N) Designed as P2P Indices

Drawbacks Advantages No keyword search Susceptible of malicious activities Scalable Fault Tolerant Common Applications: Storage systems Application-level multicasts Event Notification

Decentralized Discovery mechanisms Centralized indexes and repositories Flooding broadcast of queries Selective forwarding/routing of queries Decentralized hashing index systems Distributed indexes and repositories

Clients Content Broker Content Distribution Networks

Drawbacks Advantages Infrastructure difficult to setup Cost Cache Coherence “Slash-Dot” effect Low latency delivery of content Cuts ISP’s bandwidth costs Load balancing QoS is possible Centrally managed, pre- installed network

Conclusion Not one system fits-all Drawbacks for all systems include malicious activities Business-oriented and Research discovery mechanisms will merge Links: