Presentation is loading. Please wait.

Presentation is loading. Please wait.

Distributed Hash Tables David Tam Patrick Pang Presentation Outline What is DHT (Distributed Hash Table)? Why DHTs? Applications How lookup works? Alternatives.

Similar presentations


Presentation on theme: "Distributed Hash Tables David Tam Patrick Pang Presentation Outline What is DHT (Distributed Hash Table)? Why DHTs? Applications How lookup works? Alternatives."— Presentation transcript:

1

2 Distributed Hash Tables David Tam Patrick Pang

3 Presentation Outline What is DHT (Distributed Hash Table)? Why DHTs? Applications How lookup works? Alternatives to DHTs Performance – Routing Performance – Load Balancing Security – Routing Attack Security – Inconsistent Behaviour Comparison to Other Facilities Current Research Projects Conclusion

4 What is DHT? Distributed hash table Distributed application get (key) data node …. put(key, data) (Figure adopted from Frans Kaashoek) DHT provides the information look up service for P2P applications. Nodes uniformly distributed across key space Nodes form an overlay network Nodes maintain list of neighbours in routing table Decoupled from physical network topology

5 Why DHTs? Why Do We Need DHTs? Simplifies the development for large-scale distributed Apps Better security and robustness Simple API Exploits P2P resources Why Middleware? Simplifies the development for large-scale distributed Apps Better security and robustness Simple API

6 Applications Anything that requires a hash table Databases, FSes, storage, archival Web serving, caching Content distribution Query & indexing Naming systems Communication primitives Chat services Application-layer multi-casting Event notification services Publish/subscribe systems ?

7 How lookup works? startintervalsucc. 3[3,4)5 4[4,6)5 6[6,10)7 10[10,2)10 Finger Table for Node Example: Chord [Stoica et. al.]

8 How lookup works? startintervalsucc. 11[11,12)12 [12,14)12 14[14,2)14 2[2,10)2 Finger Table for Node 10 1 Example: Chord

9 How lookup works? startintervalsucc. 11[11,12)12 [12,14)12 14[14,2)14 2[2,10)2 Finger Table for Node 10 1 Example: Chord

10 How lookup works? startintervalsucc. 15[15,0)15 0[0,2)1 2[2,6)2 6[6,13)7 Finger Table for Node 14 Example: Chord

11 How lookup works? startintervalsucc. 15[15,0)15 0[0,2)1 2[2,6)2 6[6,13)7 Finger Table for Node 14 Example: Chord

12 How lookup works? Now Node 2 can retrive information for key 0 from Node 1. 1 Example: Chord

13 Alternatives to DHTs Distributed file system Centralized lookup P2P flooding queries Server Client Internet Server (Figures adopted from Frans Kaashoek) N4N4 Target Start N6N6 N9N9 N7N7 N8N8 N3N3 N2N2 N1N1 N 10 N4N4 Target Start N6N6 N9N9 N7N7 N8N8 N3N3 N2N2 N1N1 N 10 DB

14 Performance -- Lookup Purpose -- to locate a target node Each step, try to get closer to locating target node Ask a closer neighbour Performance & scalability tied directly to lookup algorithm 2 Aspects to Scalability size of routing table – O(log N) lookup path length – O(log N) 2 Aspects to Performance Path latency Lookup path length (# hops) 3 Techniques proximity lookup proximity neighbour selection geographic layout

15 Performance -- Load Balancing Issues Hot-spots Content Lookup Heterogeneous nodes & paths System flux Solution Replication is the key Also good for fault-tolerance Cache lookup answers backwards along path

16 Security – Incorrect Lookup (1) When asked for the next hop, give a wrong answer startintervalsucc. 3[3,4)5 4[4,6)5 6[6,10)7 10[10,2)10 Finger Table for Node 2 Node 2 to Node 10: Please tell me how to reach key 0 …

17 Security – Incorrect Lookup (2) When asked for the next hop, give a wrong answer startintervalsucc. 11[11,12)12 [12,14)12 14[14,2)14 2[2,10)2 Finger Table for Node Node 2 to Node 10: Please tell me how to reach key 0 …. Node 10 answers: ask Node 14

18 When asked for the next hop, give a wrong answer startintervalsucc. 15[15,0)15 0[0,2)1 2[2,6)2 6[6,13)7 Finger Table for Node Node 2 to Node 14: Please tell me how to reach key 0 …. Node 14 answers: ask Node 10 Security – Incorrect Lookup (3) 0

19 Security – Incorrect Lookup (4) Solution [Sit and Morris]: Define verifiable system invariant Allow the querier to observe lookup progress Our idea how this can be implemented: Concretely, using an integral monotonically decreasing quantity to implement the idea of progress. The concept of monotonically decreasing quantity has been used in program construction guaranteeing total correctness. [Parnas]

20 Security – Inconsistent Behaviour Inconsistent Behaviour, i.e., lie intelligibly Sybil attack [Kaashoek] Solution 1: public key solution

21 Security – Inconsistent Behaviour Inconsistent Behaviour, i.e., lie intelligibly Sybil attack [Kaashoek] Solution 1: public key solution Solution 2: Byzantine Protocol Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport]

22 Inconsistent Behaviour, i.e., lie intelligibly Sybil attack [Kaashoek] Security – Inconsistent Behaviour Solution 1: public key solution Solution 2: Byzantine Protocol Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport] Commander Lieutenant 1Lieutenant 2 he said retreat attack

23 Security – Inconsistent Behaviour Inconsistent Behaviour, i.e., lie intelligibly Sybil attack [Kaashoek] Solution 1: public key solution Solution 2: Byzantine Protocol Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport] Commander Lieutenant 1Lieutenant 2 retreat attack he said retreat

24 Comparison to Other Facilities FacilityAbstractionEasy Use/PrgScalabilityLoad-Balance DHThigh yes Centralized Lookupmedium lowno P2P flooding queriesmediumhighlowno Distributed FSlowmedium no FacilityFault-ToleranceSelf-OrgAdmin DHThighyeslow Centralized Lookuplownomedium P2P flooding queriesdependsyeslow Distributed FSmediumnohigh

25 Research Projects Iris – security & fault-tolerance – US Govt Chord – circular key space Pastry – circular key space Tapestry – hypercube space CAN – n-dimensional key space Kelips – n-dimensional key space DDS-- middleware platform for internet service construction -- cluster-based -- incremental scalability

26 Summary Good middleware platform Exploits P2P networks An exciting new research area

27 References Lamport, Leslie et. al. The Byzantine Generals Problem Sit, Emil, Morris, Robert. Security Considerations for Peer- to-Peer Distributed Hash Tables Kaashoek, Frans. Distributed Hash Tables – Building large- sacle, robust distributed applications Stoica, Ion et. al. Chord: A scalable peer-to-peer lookup service for Internet applications Parnas, D. L. Connecting Theory to Practice: Software Engineering Programme


Download ppt "Distributed Hash Tables David Tam Patrick Pang Presentation Outline What is DHT (Distributed Hash Table)? Why DHTs? Applications How lookup works? Alternatives."

Similar presentations


Ads by Google