Presentation is loading. Please wait.

Presentation is loading. Please wait.

Intel Research Timothy Roscoe P2: Implementing Declarative Overlays Timothy Roscoe Boon Thau Loo, Tyson Condie, David Gay, Joseph M. Hellerstein, Petros.

Similar presentations


Presentation on theme: "Intel Research Timothy Roscoe P2: Implementing Declarative Overlays Timothy Roscoe Boon Thau Loo, Tyson Condie, David Gay, Joseph M. Hellerstein, Petros."— Presentation transcript:

1 Intel Research Timothy Roscoe P2: Implementing Declarative Overlays Timothy Roscoe Boon Thau Loo, Tyson Condie, David Gay, Joseph M. Hellerstein, Petros Maniatis, Ion Stoica Intel Research at Berkeley UC Berkeley P2

2 Timothy Roscoe Intel Research 2 Overlays: a broad view Overlay: the routing and message forwarding component of any non-trivial distributed system Internet Overlay

3 Timothy Roscoe Intel Research 3 Overlays Everywhere… Many examples: Many examples: Internet Routing, multicast Internet Routing, multicast Content delivery, file sharing, DHTs, Google Content delivery, file sharing, DHTs, Google Microsoft Exchange Microsoft Exchange Tibco (technology interoperation) Tibco (technology interoperation) Overlays are a fundamental tool for repurposing communication infrastructures Overlays are a fundamental tool for repurposing communication infrastructures Get a bunch of friends together and build your own ISP (Internet evolvability) Get a bunch of friends together and build your own ISP (Internet evolvability) You dont like Internet Routing? Make up your own rules (RON) You dont like Internet Routing? Make up your own rules (RON) Paranoid? Run Freenet Paranoid? Run Freenet Intrusion detection with friends (DDI, Polygraph) Intrusion detection with friends (DDI, Polygraph) Have your assets discover each other (iAMT) Have your assets discover each other (iAMT) Internet Overlay Distributed systems innovation needs overlays

4 Timothy Roscoe Intel Research 4 If only it werent so hard In theory In theory Figure out right properties Figure out right properties Get the algorithms and protocols Get the algorithms and protocols Implement them Implement them Tune them Tune them Test them Test them Debug them Debug them Repeat Repeat But in practice But in practice No global view No global view Wrong choice of algorithms Wrong choice of algorithms Incorrect implementation Incorrect implementation Pathological timeouts Pathological timeouts Partial failures Partial failures Impaired introspection Impaired introspection Homicidal boredom Homicidal boredom Next to no debug support Next to no debug support Its hard enough as it is Do I also need to reinvent the wheel every time?

5 Timothy Roscoe Intel Research 5 Our Goal Make network development more accessible to developers of distributed applications Make network development more accessible to developers of distributed applications Specify network at a high-level Specify network at a high-level Automatically translate specification into executable Automatically translate specification into executable Hide everything they dont want to touch Hide everything they dont want to touch Enjoy performance that is good enough Enjoy performance that is good enough Do for networked systems what SQL and the relational model did for databases Do for networked systems what SQL and the relational model did for databases

6 Timothy Roscoe Intel Research 6 The argument: The set of routing tables in a network represents a distributed data structure The set of routing tables in a network represents a distributed data structure The data structure is characterized by a set of ideal properties which define the network The data structure is characterized by a set of ideal properties which define the network Thinking in terms of structure, not protocol Thinking in terms of structure, not protocol Routing is the process of maintaining these properties in the face of changing ground facts Routing is the process of maintaining these properties in the face of changing ground facts Failures, topology changes, load, policy… Failures, topology changes, load, policy…

7 Timothy Roscoe Intel Research 7 Routing as Query Processing In database terms, the routing table is a view over changing network conditions and state In database terms, the routing table is a view over changing network conditions and state Maintaining it is the domain of distributed continuous query processing Maintaining it is the domain of distributed continuous query processing Not merely an analogy: We have implemented a general routing protocol engine as a query processor. Not merely an analogy: We have implemented a general routing protocol engine as a query processor.

8 Timothy Roscoe Intel Research 8 Two directions 1.Declarative expression of Internet Routing protocols Loo et. al., ACM SIGCOMM 2005Loo et. al., ACM SIGCOMM Declarative implementation of overlay networks Loo et. al., ACM SOSP 2005Loo et. al., ACM SOSP 2005 The focus of this talk (and my work)The focus of this talk (and my work)

9 Timothy Roscoe Intel Research 9 P2: A Declarative Overlay Engine Distributed state Distributed state Distributed soft state in relational tables, holding tuples of values Distributed soft state in relational tables, holding tuples of values route (S, D, H) route (S, D, H) Non-stored information passes around as event tuple streams Non-stored information passes around as event tuple streams message (X, D) message (X, D) Overlay specification in declarative logic language (OverLog) Overlay specification in declarative logic language (OverLog) :-,, …,. :-,, …,. Location place individual tuples at specific nodes Location place individual tuples at specific nodes D) :- D, H), D). D) :- D, H), D). (a, x, c) (a, z, f) (a, z, t) z) z) z)

10 Timothy Roscoe Intel Research 10 P2 Dataflow Overlog automatically translated to dataflow graph Overlog automatically translated to dataflow graph C++ dataflow elements (similar to Click elements) C++ dataflow elements (similar to Click elements) Implements: Implements: relational operators (joins, selections, projections) relational operators (joins, selections, projections) flow operators (multiplexers, demultiplexers, queues) flow operators (multiplexers, demultiplexers, queues) network operators (congestion control, retry, rate limits) network operators (congestion control, retry, rate limits) Interlinked via asynchronous push or pull typed flows Interlinked via asynchronous push or pull typed flows Engine executes dataflow graph at runtime Engine executes dataflow graph at runtime A distributed query processor to maintain overlays demux

11 Timothy Roscoe Intel Research 11 Example: Ring Routing Every node has an address (e.g., IP address) and an identifier (large random) Every node has an address (e.g., IP address) and an identifier (large random) Every object has an identifier Every object has an identifier Order nodes and objects into a ring by their identifiers Order nodes and objects into a ring by their identifiers Objects served by their successor node Objects served by their successor node Every node knows its successor on the ring Every node knows its successor on the ring To find object K, walk around the ring until I locate Ks immediate successor node To find object K, walk around the ring until I locate Ks immediate successor node

12 Timothy Roscoe Intel Research 12 Example: Ring Routing How do I find the responsible node for a given key k? n.lookup(k) if k in (n, n.successor) return n.successor else return n.successor. lookup(k)

13 Timothy Roscoe Intel Research 13 Ring State n.lookup(k) if k in (n, n.successor) return n.successor else return n.successor. lookup(k) Node state tuples node(NAddr, N) successor(NAddr, Succ, SAddr) Transient event tuples lookup (NAddr, Req, K)

14 Timothy Roscoe Intel Research 14 Pseudocode to OverLog n.lookup(k) if k in (n, n.successor] return n.successor else return n.successor. lookup(k) Node state tuples node(NAddr, N) successor(NAddr, Succ, SAddr) Transient event tuples lookup (NAddr, Req, K) (Req, K, SAddr) :- (NAddr, Req, K), node (NAddr, N), succ (NAddr, Succ, SAddr), K in (N, Succ].

15 Timothy Roscoe Intel Research 15 Pseudocode to OverLog n.lookup(k) if k in (n, n.successor] return n.successor else return n.successor. lookup(k) Node state tuples Node (NAddr, N) Successor NAddr, Succ, SAddr) Transient event tuples lookup (NAddr, Req, K) (Req, K, SAddr) :- (NAddr, Req, K), node (NAddr, N), succ (NAddr, Succ, SAddr), K in (N, Succ]. (SAddr, Req, K) :- (NAddr, Req, K), node (NAddr, N), succ (NAddr, Succ, SAddr), K not in (N, Succ].

16 Timothy Roscoe Intel Research 16 Location Specifiers n.lookup(k) if k in (n, n.successor] return n.successor else return n.successor. lookup(k) Node state tuples node(NAddr, N) successor(NAddr, Succ, SAddr) Transient event tuples lookup (NAddr, Req, K) R1 K, SAddr) :- Req, K), N), Succ, SAddr), K in (N, Succ]. R2 Req, K) :- Req, K), N), Succ, SAddr), K not in (N, Succ].

17 Timothy Roscoe Intel Research 17 Implementation: From OverLog to Dataflow Traditional problem in databases Traditional problem in databases Turn logic into relational algebra Turn logic into relational algebra Joins, projections, selections, aggregations, etc. Joins, projections, selections, aggregations, etc.

18 Timothy Roscoe Intel Research 18 From OverLog to Dataflow K, SI) : - R, K), N), S, SI), K in (N, S]. R, K) :- R, K), N), S, SI), K not in (N, S].

19 Timothy Roscoe Intel Research 19 From OverLog to Dataflow R1 K, SI) : - R, K), N), S, SI), K in (N, S]. R2 R, K) :- R, K), N), S, SI), K not in (N, S].

20 Timothy Roscoe Intel Research 20 From OverLog to Dataflow R1 K, SI) : - R, K), N), S, SI), K in (N, S]. R2 R, K) :- R, K), N), S, SI), K not in (N, S].

21 Timothy Roscoe Intel Research 21 From OverLog to Dataflow R1 K, SI) : - R, K), N), S, SI), K in (N, S]. R2 R, K) :- R, K), N), S, SI), K not in (N, S].

22 Timothy Roscoe Intel Research 22 From OverLog to Dataflow R1 K, SI) : - R, K), N), S, SI), K in (N, S]. R2 R, K) :- R, K), N), S, SI), K not in (N, S].

23 Timothy Roscoe Intel Research 23 From OverLog to Dataflow R1 K, SI) : - R, K), N), S, SI), K in (N, S]. R2 R, K) :- R, K), N), S, SI), K not in (N, S].

24 Timothy Roscoe Intel Research 24 From OverLog to Dataflow R1 K, SI) : - R, K), N), S, SI), K in (N, S]. R2 R, K) :- R, K), N), S, SI), K not in (N, S].

25 Timothy Roscoe Intel Research 25 From OverLog to Dataflow R1 K, SI) : - R, K), N), S, SI), K in (N, S]. R2 R, K) :- R, K), N), S, SI), K not in (N, S].

26 Timothy Roscoe Intel Research 26 From OverLog to Dataflow One rule strand per OverLog rule Rule order is immaterial Rule strands could execute in parallel

27 Timothy Roscoe Intel Research 27 From OverLog to Dataflow nodesucc Rule R1 lookup Rule R2... D e m u x Q u e u e U D P T x U D P R x C C R x Sched Q u e u e... C C T x

28 Timothy Roscoe Intel Research 28 Implementation Elements are C++ objects Elements are C++ objects Reference-counted immutable tuples Reference-counted immutable tuples Fast tuple hand-off Fast tuple hand-off ~50 ia32 instructions, ~300 cycles ~50 ia32 instructions, ~300 cycles Currently single-threaded Currently single-threaded Select loop, timers, etc. Select loop, timers, etc. Element state stored in tables Element state stored in tables C.f. database catalogues: reuse data model wherever appropriate C.f. database catalogues: reuse data model wherever appropriate Conventional Bison/Flex parser Conventional Bison/Flex parser

29 Timothy Roscoe Intel Research 29 It actually works. For instance, we implemented Chord in P2 For instance, we implemented Chord in P2 Popular distributed hash table Popular distributed hash table Complex overlay Complex overlay Dynamic maintenance Dynamic maintenance How do we know it works? How do we know it works? Same high-level properties Same high-level properties Logarithmic overlay diameter Logarithmic overlay diameter Logarithmic state size Logarithmic state size Consistent routing with churn Consistent routing with churn Comparable performance to hand-coded implementations Comparable performance to hand-coded implementations

30 Timothy Roscoe Intel Research 30 Key point: remarkably concise overlay specification Full specification of Chord overlay, including Full specification of Chord overlay, including Failure recovery Failure recovery Multiple successors Multiple successors Stabilization Stabilization Optimized maintenance Optimized maintenance 44 OverLog rules 44 OverLog rules And it runs! And it runs! 10 pt font

31 Timothy Roscoe Intel Research 31 Comparison: MIT Chord in C++

32 Timothy Roscoe Intel Research 32 Lookup length in hops

33 Timothy Roscoe Intel Research 33 Maintenance bandwidth (comparable with MIT Chord)

34 Timothy Roscoe Intel Research 34 Latency without churn

35 Timothy Roscoe Intel Research 35 Latency under churn Compare with Bamboo non- adaptive timeout figures…

36 Timothy Roscoe Intel Research 36 Consistency under churn

37 Timothy Roscoe Intel Research 37 The story so far: Can specify overlays as continuous queries in a logic language Can specify overlays as continuous queries in a logic language Compile to a graph of dataflow elements Compile to a graph of dataflow elements Efficiently execute graph to perform routing and forwarding Efficiently execute graph to perform routing and forwarding Overlays exhibit similar performance characteristics Overlays exhibit similar performance characteristics But … Once you have a distributed query processor, lots of things fall off the back of the truck…

38 Timothy Roscoe Intel Research 38 What else does this buy you? Introspection (w/ Atul Singh, Rice) Overlay invariant monitoring: a distributed watchpoint Overlay invariant monitoring: a distributed watchpoint Whats the average path length? Whats the average path length? Is routing consistent? Is routing consistent? Execution tracing at pseudo-code granularity: logical stepping Execution tracing at pseudo-code granularity: logical stepping Why did rule R7 trigger? Why did rule R7 trigger? … and at dataflow granularity: intermediate representation stepping … and at dataflow granularity: intermediate representation stepping Why did that tuple expire? Why did that tuple expire? Great way to do distributed debugging and logging Great way to do distributed debugging and logging In fact, we use it and have found a number of bugs… In fact, we use it and have found a number of bugs…

39 Timothy Roscoe Intel Research 39 What else does this buy you? 2. Transport reconfiguration Dataflow paradigm thins out layer boundaries Dataflow paradigm thins out layer boundaries Mix and match transport facilities (retries, congestion control, rate limitation, buffering) Mix and match transport facilities (retries, congestion control, rate limitation, buffering) Spread bits of transport through the application to suit application requirements Spread bits of transport through the application to suit application requirements Automatically! Automatically!

40 Timothy Roscoe Intel Research 40 In fact, a rich seam for future research… Reconfigurable transport protocols Reconfigurable transport protocols Debugging and logging support Debugging and logging support The right language – global invariants The right language – global invariants Use distributed joins as abstraction mechanism Use distributed joins as abstraction mechanism Optimization techniques Optimization techniques Inc. multiquery optimization Inc. multiquery optimization Monitoring other distributed systems and networks Monitoring other distributed systems and networks Evolve towards more general query processor? Evolve towards more general query processor? PIER heritage returns PIER heritage returns

41 Timothy Roscoe Intel Research 41 Summary Overlays enable distributed system innovation Overlays enable distributed system innovation Wed better make them easier to build, reuse, understand Wed better make them easier to build, reuse, understand P2 enables P2 enables High-level overlay specification in OverLog High-level overlay specification in OverLog Automatic translation of specification into dataflow graph Automatic translation of specification into dataflow graph Execution of dataflow graph Execution of dataflow graph Explore and Embrace the trade-off between fine-tuning and ease of development Explore and Embrace the trade-off between fine-tuning and ease of development Get the full immersion treatment in our paper in SOSP 05, code release imminent Get the full immersion treatment in our paper in SOSP 05, code release imminent

42 Timothy Roscoe Intel Research 42 Thanks! Questions? A few to get you started: A few to get you started: Who cares about overlays? Who cares about overlays? Logic? You mean Prolog? Eeew! Logic? You mean Prolog? Eeew! This language is really ugly. Discuss. This language is really ugly. Discuss. But what about security? But what about security? Is anyone ever going to use this? Is anyone ever going to use this? Is this as revolutionary and inspired as it looks? Is this as revolutionary and inspired as it looks?


Download ppt "Intel Research Timothy Roscoe P2: Implementing Declarative Overlays Timothy Roscoe Boon Thau Loo, Tyson Condie, David Gay, Joseph M. Hellerstein, Petros."

Similar presentations


Ads by Google