1. CoDoNs A High-Performance Alternative for the Domain Name System Emin Gün Sirer Venugopalan Ramasubramanian Computer Science, Cornell University

2. introduction caching is widely-used to improve latency and to decrease overhead passive caching caches distributed throughout the network store objects that are encountered not well-suited for a large-class applications

3. problems with passive caching no performance guarantees heavy-tail effect large percentage of queries to unpopular objects ad-hoc heuristics for cache management introduces coherency problems difficult to locate all copies weak consistency model

4. domain name system (dns) part of critical Internet infrastructure resolves names to addresses “www.cnn.com”  67.45.223.12 also resolves other flat queries hostname to mail server mappings hostname to owner information ipaddress to hostname and others based on a static hierarchy of servers

5. dns operation root edumilgovcomorg cornellupenn cs math ecearts wwwsyslab namespace partitioned into hierarchy lookup occurs from the top down through delegations static, manually-crafted, fragile hierarchy lookups are expensive no bounds! cnn ns1.verisign.com a.root-servers.net cit.cornell.edu ns.cs.cornell.edu

6. dns band-aids root edumilgovcomnet cornellupenn cs math ececitcit wwwsyslab replicate nodes at the top of the hierarchy 13 root servers 13 com, mil servers 2 org servers use passive caching for dns records at every possible location associate a TTL with every dns record no invalidations

7. dns problems performance bottleneck cold misses lead to high latencies vulnerable to DDoS attacks November 2002 attack on root servers can target branches of the name tree high load on nodes close to the root tree not a scalable data structure manual maintenance expensive, error prone DNS hierarchy full of errors coherency guarantees at odds with performance low TTL enables dynamic updates reduces performance

8. a better solution? distributed hash tables (DHTs) can help self-organizing scalable worst-case performance bounds but average-case performance for scalable DHTs is too high

9. 0122 prefix-matching DHTs object 0121 0121 = hash(“www.cnn.com”) 2012 0021 0112 map all nodes into an identifier space map all objects into same space based on key log b N hops several RTTs on the Internet

10. DHT properties scalable DHTs not suited for DNS Morris, Balakrishnan et al. O(log N) overlay hops is too slow for critical, latency-sensitive applications need faster lookups proactive, model-driven caching can help

11. Outline Problem passive caching DNS: a motivating example DHTs Beehive approach analytical model distributed implementation evaluation CoDoNs overview & operation evaluation Summary

12. overview of beehive general replication framework suitable for structured DHTs decentralization, self-organization, resilience properties high performance: O(1) average lookup time scalable: minimize number of replicas and reduce storage, bandwidth, and network load adaptive: promptly respond to changes in popularity – flash crowds

13. key intuition tunable latency adjust extent of replication for each object fundamental space- time tradeoff 2012 0021 0112 0122

14. analytical model optimization problem minimize: total number of replicas, s.t., average lookup performance  C configurable target lookup performance continuous range, sub one-hop minimizing number of replicas decreases storage and bandwidth overhead

15. target domain zipf-like query distributions number of queries to r th popular object  1/r  fraction of queries for m most popular objects  (m 1-  - 1) / (M 1-  - 1) commonly encountered in practice dns, web  typically between 0.80 and 1.00 MIT DNS trace = 0.91 traceDecUPisaFuNetUCBQuestNLANR  0.830.84 0.830.880.90

16. level of replication nodes share i prefix-digits with the object i hop lookup latency lower level means greater extent of replication replicated on N/b i nodes

17. optimization problem minimize (storage/bandwidth) x 0 + x 1 /b + x 2 /b 2 + … + x K-1 /b K-1 such that (average lookup time is C hops) K – (x 0 1-  + x 1 1-  + x 2 1-  + … + x K-1 1-  )  C and x 0  x 1  x 2  …  x K-1  1 b: base K: log b (N) x i : fraction of objects replicated at level i

18. optimal closed-form solution d j (K’ – C) 1 + d + … + d K’-1 1 1 -  [ ] x* i =, 0  i  K’ – 1 where d = b (1-  ) / , K’  i  K K’ is determined by setting x* K’-1  1  d K’-1 (K’ – C) / (1 + d + … + d K’-1 )  1 1

19. implications O(1) average-case lookup performance C hops for Zipf-like distributions bounded worst-case K’ forms upper bound optimal number of replicas for   1 no wasted bandwidth, storage or coordination

20. latency - overhead trade off

21. beehive: system overview estimation popularity of objects, zipf parameter local measurement, limited aggregation replication apply analytical model independently at each node push new replicas to nodes at most one hop away

22. beehive replication protocol 0 1 2 * home node EL 3 0 1 * EBIL 2 0 * ABCDEFGHI L 1 object 0121

23. replication protocol low-overhead mechanisms distributed, no consensus asynchronous performed in the background modest state per object id (128 bits) home node indicator (1 bit) version number (64 bits)

24. mutable objects leverage the underlying structure of DHT replication level indicates the locations of all the replicas proactive propagation to all nodes from the home node home node sends to one-hop neighbors with i matching prefix-digits level i nodes send to level i+1 nodes

25. implementation and evaluation implemented using Pastry as the underlying DHT evaluation using a real DNS workload MIT DNS trace 1024 nodes, 40960 objects compared with passive caching on pastry main properties evaluated lookup performance storage and bandwidth overhead adaptation to changes in query distribution

26. 0816243240 0 0.5 1 1.5 2 2.5 3 time (hours) latency (hops) Pastry PC-Pastry Beehive evaluation: lookup performance passive caching is not very effective because of heavy tail query distribution and mutable objects. beehive converges to the target of 1 hop

27. evaluation: overhead 0816243240 0 0.5 1 1.5 2 x 10 6 time (hours) object transfers (#) PC-Pastry Beehive Object Transfers Storage average number of replicas per node Pastry40 PC-Pastry420 Beehive380

28. evaluation: flash crowds 32404856647280 0 0.5 1 1.5 2 2.5 3 Latency time (hours) latency (hops) Pastry PC-Pastry Beehive

29. evaluation: zipf parameter change

30. Outline Problem passive caching DNS: a motivating example DHTs Beehive approach analytical model distributed implementation evaluation CoDoNs overview & operation evaluation Summary

31. Cooperative Domain Name System (CoDoNS) replacement for legacy DNS peer-to-peer DNS based on Beehive incremental deployment path completely transparent to clients uses legacy DNS to populate resource records on demand can operate without legacy DNS deployed on planet-lab

32. CoDoNs operation home node initially populates CoDoNs with binding from legacy DNS CoDoNs will propagate the optimal number of replicas Upon DNS record expiration, the home node checks binding for change www.cnn.com

33. CoDoNs lookup performance

34. CoDoNSLegacy DNS median6.56 ms38.8 ms 90 th percentile281 ms337 ms

35. CoDoNs load balancing CoDoNs automatically balances load

36. CoDoNs security not an issue in a single administration domain e.g. akamai, google, msn, etc. attacks targeted at the DHT Castro et al. ’02 work on secure DHTs attacks targeted at Beehive outlier elimination limited impact attacks targeted at CoDoNs DNSSEC signatures

37. dns trickery server-side computation akamai and similar companies reroute clients to nearest nodes through dns dynamically computed records with low ttl not compatible with caching CoDoNs supports this hack by not caching entries whose TTL is lower than a threshold but better to perform server selection on the client side or not at all… CoDoNs already provides high average case performance and load balancing

38. CoDoNs implications name delegations can be purchased and propagated independently of server setup no static tree naming hierarchy independent of physical server hierarchy no manual configuration misconfiguration and broken delegations unlikely no ttl push updates at any time no verisign

39. advantages of CoDoNS high performance low lookup latency median latency of 7 ms for codons (planet-lab), 39 ms for legacy DNS resilience against denial of service attacks load balancing around hotspots self configuring around host and network failures fast, coherent updates no TTLs, updates can be propagated at any time

40. related work - DHTs Prefix-based Plaxton, Pastry, Chord, Tapestry O(log N) lookup, O(log N) storage deBruijn graphs Koorde, [Wieder & Naor 03] O(log N/log log N) lookup, O(log N) storage Butterfly Viceroy: O(log N) lookup, O(1) storage Farsite O(d) lookup, O(dn 1/d ) storage Kelips Kelips: O(1) lookup, O(  N) storage Full knowledge [Gupta, Liskov, Rodrigues]: O(1) lookup, O(N) storage

41. related work - DNS traces and zipf distributions web caching and zipf-like distributions: evidence and implications  Breslau, Cao, Fan, Phillips, and Shenker [infocom’99] popularity of gnutella queries and their implications on scalability  Sripanidkulchai [2002] caching and replication replication strategies in unstructured p2p networks  Cohen and Shenker [sigcomm’02] cup: controlled update propagation in p2p networks  Roussopoulos and Baker [usenix’03] DNS development of DNS system  Mockapetris [sigcomm’88] DNS performance and effectiveness of caching  Jung, Sit, Balakrishnan, and Morris [sigmetrics’01] serving DNS using a peer-to-peer lookup service  Cox, Muthitacharoen, and Morris [iptps’02]

42. conclusions model-driven proactive caching O(1) lookup performance with optimal replica cost beehive: a general replication framework structured overlays with uniform fan-out high performance, resilience, improved availability well-suited for latency sensitive applications CoDoNs: a modern replacement for DNS http://www.cs.cornell.edu/people/egs/beehive/

44. evaluation: zipf parameter change

45. evaluation: instantaneous bandwidth overhead 32404856647280 0 5 10 15 20 Object Transfers per sec time (hours) object transfers per sec PC-Pastry Beehive

46. typical values of zipf parameter MIT DNS trace:  = 0.91 Web traces: traceDecUPisaFuNetUCBQuestNLANR  0.830.84 0.830.880.90

47. comparative overview of structured DHTs DHTlookup performance CANO(dN 1/d ) Chord, Kademlia, Pastry, Tapestry, Viceroy O(logN) de Bruijn graphs (Koorde)O(logN/loglogN) Kelips, Salad, [Gupta, Liskov, Rodriguez] O(1)

48. Beehive DNS: Lookup Performance CoDoNSLegacy DNS median6.56 ms38.8 ms 90 th percentile281 ms337 ms

49. introduction distributed peer-peer overlay networks decentralized self-organized distributed hash tables (DHTs) store – lookup interface unstructured DHTs Freenet, Gnutella, Kazaa bad lookup performance: accuracy and latency

50. example b = 32 C = 1  = 0.9 N = 10,000 M = 1,000,000 x* 0 = 0.001102 = 1102 objects x* 1 = 0.0519 = 51900 objects x* 2 = 1 total storage = 3700 objects per node total storage for Kelips = M/  N = 10,000 objects per node

51. structured overlays distributed peer-to-peer overlays decentralized, self-organized, resilient structured DHTs object storage and retrieval bounded average, worst-case latency latency sensitive applications domain name service (DNS) and web access need sub one hop latency

52. analytical model configurable target lookup performance continuous range even better with proximity routing minimizing number of replicas provides storage as well as bandwidth efficiency k’ is a upper bound on lookup performance of successful query assumptions homogeneous object sizes infrequent updates

53. beehive replication protocol periodic packets to nodes in routing table asynchronous and independent exploit structure of underlying DHT replication packet sent by node A to each node B in level i of routing table node B pushes new replicas to A and tells A which replicas to remove fluctuations in estimated popularity aging to prevent sudden changes hysteresis to limit thrashing

54. evaluation: DNS application DNS survey queried 594059 unique domain names TTL distribution: 95% < 1 day rate of change of entries: 0.13% per day MIT DNS trace: 4 ~ 11 december 2000 4 million queries for 300,000 distinct names zipf parameter: 0.91 setup simulation mode on single node 1024 nodes, 40960 distinct objects 7 queries per sec from MIT trace 0.8% per day rate of change

55. introduction lookup latency and storage CANChordPastryKelips latencyO(dN 1/d )O(log 2 N)O(log b N)O(1) 1,000,000 nodes 39.8 hops d = 10 19.93 hops4.98 hops b = 16 ~1 hop 86.4 ms/hop 3.4 sec1.7 sec0.43 sec0.0864 sec storageO(1/N) O(1/  N)

56. improving lookup latency passive caching of lookup results  not effective for heavy-tail query distributions  no guaranteed performance  updates invalidate cached results O(1) lookup performance trade off storage and bandwidth for performance Kelips: O(  N) replicas per object [GLR2003]: complete routing table

57. levels of replication 37420 ***** ***** level 0 3**** 3**** level 1 37** *level 2 obj id

58. optimal storage: C = 1 hop

59. summary and useful properties constant average lookup latency the constant is configurable popular objects have shorter lookup times? upper bounded by K’ (2 for  = 0.8) optimal overhead the storage and bandwidth requirements can be estimated overhead decreases with increasing  high availability for popular objects mitigates flash crowd effect proactive replication supports mutable objects more benefits can be derived by using proximity optimizations

60. a peer-peer DNS why p2p? iterative queries name-server mis-configurations  lots of failures and increased traffic less availability  chain of NS records update problem (Akamai) why BeeHive? constant lookup time  upper bound given by K’ (~2 or 3 hops)  comparable or better performance better availability due to replication support for mutability

61. DNS over beehive: distributed cooperative active cache deploy incrementally non-uniform rate of change scale popularity metric proportionately lookup failure negative caching reverse iterative resolution  lookup x.y.com, then y.com, then com…  fetches NS records  locality  inverse queries

62. DNS over BeeHive: security DNSSEC public key cryptography, signature chains namespace management  big sizes of key and sig records cache chain of key records for authentication popularity(y.com) > popularity(x.y.com) avoid duplicate key records authenticated denial reverse iterative resolution

63. other potential applications translation services semantic free lookup P6P p2p file sharing  text based search  anti-social applications? web  widely varying object sizes  dynamic content

64. conclusions BeeHive: p2p system based on differential replication goals efficient: constant lookup time with minimal overhead robust: self-organization and resilience against the vagaries of the network secure: resilience against malicious elements CoDoNS: Cooperative DNS on BeeHive

65. notations b:base of the DHT system N:number of nodes (b K ) M:number of objects  :alpha of zipf-like query distribution x j :fraction of objects replicated at level j or lower x 2 = fraction of objects replicated at levels 0, 1, and 2 0  x 0  x 1  x 2  …  x K-1  x K = 1

66. storage and bandwidth per node storage required at level j = M(x j – x j-1 )/b j total per node storage = Mx 0 + M(x 1 – x 0 )/b + M(x 2 – x 1 )/b 2 + … + M(x K – x K-1 )/b K = M [(1 – 1/b)(x 0 + x 1 /b + x 2 /b 2 + … + x K-1 /b K-1 ) + 1/b K ] total bandwidth = M b K [(1 – 1/b)(x 0 + x 1 /b + x 2 /b 2 + … + x K-1 /b K-1 )]

67. lookup latency fraction of queries for Mx j objects  ((Mx j ) 1-  - 1) / (M 1-  - 1)  x j 1-  average lookup time at level j  j (x j 1-  - x j-1 1-  ) average lookup time  (x 1 1-  - x 0 1-  ) + 2(x 2 1-  - x 1 1-  ) + … + K(x K 1-  - x K-1 1-  )  K – (x 0 1-  + x 1 1-  + x 2 1-  + … + x K-1 1-  )

68. optimization problem minimize x 0 + x 1 /b + x 2 /b 2 + … + x K-1 /b K-1 such that (x 0 1-  + x 1 1-  + x 2 1-  + … + x K-1 1-  )  K – C and x 0  x 1  x 2  …  x K-1  1 and x K-1  1

69. solution minimize x 0 + x 1 /b + x 2 /b 2 + … + x K’-1 /b K ’ -1 such that (x 0 1-  + x 1 1-  + x 2 1-  + … + x K’-1 1-  )  K’ – C solution using lagrange multiplier technique d j (K’ – C) 1 + d + … + d K’-1 1 1 -  [ ] x* j = 0  j  K’ – 1 d = b (1-  ) /  x* j = 1 K’  j  K K’ is determined by setting x* K’-1  1  d K’-1 (K’ – C) / (1 + d + … + d K’-1 )  1

70. constant lookup time systems kelips and scuttlebutt trade off bandwidth and storage for performance one-hop lookup latency resilience against transient nodes O(√N) replication of all objects  expensive update propagation