1. EPFL-I&C-LSIR [P-Grid.org] Workshop on Distributed Data and Structures ’04 NCCR-MICS [IP5] presented by Anwitaman Datta Joint work with Karl Aberer and Manfred Hauswirth {Karl.Aberer, Anwitaman.Datta, Manfred.Hauswirth}@epfl.ch Route maintenance overheads in DHT overlays EPFL-I&C-LSIR [P-Grid.org] Workshop on Distributed Data and Structures’04 NCCR-MICS [IP5]

2. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 2 P2P Need XYZ Content XYZ Network  Central index  Unstructured  Flooding  Super-peer  Power-law networks … P2P is more than just File sharing or Pirate to Pirate!

3. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 3 Goals: Efficient, scalable and reliable resource discovery/name resolution in a decentralized internet scale system Content addressable network - disentangle the underlying network from the applications and services A reliable substrate for other distributed applications Semantic Web, Grid computing, Web services, large scale event notification, P2P web-search, … A Case for P2P

4. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 4 Challenges Unreliability of autonomous participants (peers) Unreliability of the communication network Lack of global knowledge and coordination Scalability and robustness (fault tolerance) Performance

5. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 5 Approach (& issues) Distributed indexing and routing in an overlay (disentangled from the underlying network) Just like the world wide web, we can realize an overlay (but more structured) on top of the internet infrastructure e.g., Distributed Hash Table/Distributed Indexing P-Grid, Chord, DKS, CAN, Pastry, Kademlia, … Peers (re-)joining/leaving the overlay => Maintenance of the overlay routes is required Self-healing while routing Flux in overlay => system operates in dynamic equlibrium Two forces: Changes in the network making routing information unusable Maintenance operations repairing routing information

6. 1 : 12, 13 01 : 5, 10 001: 9,4 1 0,1 1 : 12, 13 01 : 5,14 001: 9,4 7 0,1 1 : 6,13 01 :10,14 000: 1,7 4 2,3 1 : 8,2 01 : 3, 10 000: 1,7 9 2,3 1 : 8, 13 00 : 7,9 011: 3,10 5 4,5 1 : 2,12 00 : 9,4 011: 3,10 14 4,5 1 : 6,8 00 : 1,7 010: 5,14 10 6,7 1 : 11,12 00 : 1,9 010: 5,14 3 6,7 0 : 4,7 11 : 2,12 101: 8,13 11 8,9 0 : 1,3 11 : 2,12 101: 8,13 6 8,9 0 : 5,9 11 : 2,12 100: 6,11 13 10,11 0 : 4,9 11 : 2,12 100: 6,11 8 10,11 0 : 5,7 10 : 6,13 12 12,13,14 0 : 1,14 10 : 11,13 2 12,13,14 000001010011 01 0001 10 100101 11 ID peer identifier Prefix Routing in P-Grid 1 : 12, 13 routing table entry query(101) @ 7 Peer 9 holds keys with prefix 001, so we call, its path is 001 Replicas

7. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 7 Any distributed access structure (such as P-Grid) requires mapping of a logical ID (associated data key) to physical ID (IP address) This mapping cannot by static in the presence of dynamic IP addresses (hence requires a directory service) A very important problem for the implementation of any P2P system directory (logical ID IP address) lookup IP address P-Grid routing based on logical address Self-referential directory implemented by P-Grid routing based on logical address lookup IP address in case of failure Self-healing, self-referential directory

8. 1 : 12, 13 01 : 5, 10 001: 9,4 1 1 1 : 12, 13 01 : 5,14 001: 9,4 7 1 1 : 6,13 01 :10,14 000: 1,7 4 2,3 1 : 8,2 01 : 3, 10 000: 1,7 9 2,3 1 : 8, 13 00 : 7,9 011: 3,10 5 4,5 1 : 2,12 00 : 9,4 011: 3,10 14 4,5 1 : 6,8 00 : 1,7 010: 5,14 10 6,7 1 : 11,12 00 : 1,9 010: 5,14 3 6,7 0 : 4,7 11 : 2,12 101: 8,13 11 8,9 0 : 1,3 11 : 2,12 101: 8,13 6 8,9 0 : 5,9 11 : 2,12 100: 6,11 13 10,11 0 : 4,9 11 : 2,12 100: 6,11 8 10,11 0 : 5,7 10 : 6,13 12 12,13,14 0 : 1,14 10 : 11,13 2 12,13,14 01 000001 01 010011 10 100101 11 query(01*) @ 7 …query(0101) @ 7 (for stale entry 5, cycle -> abort) …query(1110) @ 7 (for stale entry 14, forward to 12 or 13) …query(1110) @ 12 (is offline) …query(1110) @ 13 (for stale entry 2) ……query(0010) @ 13 (forward to 5) ……query(0010) @ 5 (forward to 7) ……query(0010) @ 7 (forward to 9) ……query(0010) @ 9 (new entry for 2 found !) …query(1110) @ 2 (new entry for 14 found !) query(01*) @ 14 (finally ) 00 1 : 2,12 Stale cache Up-to-date cache ID Presently online ID Presently offnline ID Stores mappings of peers Example offline Key as 4 bits for ID (2=0010 etc.) Routing entries repaired

9. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 9 Possible strategies Eager - Correction on Use (CoU) While using a routing table, try correcting stale entries even if the present query can be routed using alternate routes (available locally). Some entries of a particular level of routing table are unusable, but other entries of the same level are still usable. Lazy - Correction on Failure (CoF) While using a routing table, try correcting stale entries only if no alternate routes for the present query is available locally. All entries of a particular level of routing table are unusable, but other levels may still be usable.

10. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 10 Performance of overlays in flux Static resilience Given a state of the network, and no more changes, how does the network perform? P-Grid, Chord, various topologies … Dealing with network churn Given flux in the network, what maintenance cost is required to maintain a certain state. e.g., Lower bound (MIT/Chord) Simulations … (many groups) Dynamic equilibrium Given any flux in the network, and any maintenance strategy, what equilibrium state will the network operate in, and what will the maintenance cost and performance in the equilibrium state be?

11. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 11 Eager recursion a.k.a. Correction on Use ‘CoU’ LHS Rate at which repair of stale routing entries occur r up changes per 1-r up queries N rec – 1 additional recursive queries Repair makes sense only if the routing entry to be repaired corresponds to an online peer A repair is possible only if recursive query succeeds RHS Rate of entries turning stale r up changes 1-p dyn probability of non-stale references (only these can turn stale) r references at each peer for each of log 2 n levels Dynamic equilibrium equation

12. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 12 Lazy Repair Strategy (Correction on Failure ‘CoF’ ) Try to rectify stale references only when none of the references in a given level are usable Not all routing entries are treated uniformly (unlike in CoU). The number of stale entries for each routing level at each peer defines the state of that level. Markovian model. Dynamic equilibrium equation determined by equating inflow and outflow for each state At dynamic equilibrium, the number of routing levels with given number of stale entries over the whole system should not change 0 ref stale 1 ref stale 2 ref stale r ref stale … repairs ID change ID change ID change ID change N.B. We distinguish stale entries from offline peers

13. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 13 Analysis vs. Simulations (Lazy recursion)

14. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 14 Overhead with varying p on

15. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 15 Contour: Zone of operation for a maximum cost (Lazy)

16. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 16 CoU (eager) vs. CoF (lazy)

17. Taxonomy of route maintenance mechanisms Our approaches Reactive strategies

18. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 18 Summary Self-referential decentralized directory with self- healing routing Dynamic equilibrium of overlay network in flux (model & analysis) Route maintenance mechanisms Correction on Use Correction on Failure Taxonomy of maintenance mechanisms

19. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 19 Other/open issues Security/DDoS/… Identity/Authentication Authorization/Privacy Reputation/Trust Quorums/Web-of-trust Garbage collection of references Generic analysis (for various DHTs) Sensor networks or MANETs and overlays

20. Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 20 References Efficient, self-contained handling of identity in Peer-to-Peer systems, Karl Aberer, Anwitaman Datta, Manfred Hauswirth; IEEE Transactions on Knowledge and Data Engineering 16(7), July 2004IEEE Transactions on Knowledge and Data Engineering & other papers @ http://www.p-grid.org Questions?