Presentation is loading. Please wait.

Presentation is loading. Please wait.

Naiad: A Timely Dataflow System Derek G. Murray Michael Isard Frank McSherry Paul Barham Rebecca Isaacs Martín Abadi Microsoft Research 1.

Similar presentations


Presentation on theme: "Naiad: A Timely Dataflow System Derek G. Murray Michael Isard Frank McSherry Paul Barham Rebecca Isaacs Martín Abadi Microsoft Research 1."— Presentation transcript:

1 Naiad: A Timely Dataflow System Derek G. Murray Michael Isard Frank McSherry Paul Barham Rebecca Isaacs Martín Abadi Microsoft Research 1

2 Batch processing Stream processing Graph processing Timely dataflow

3 In ⋈ z? ⋈ max ⋈

4 In ⋈ z? ⋈ max ⋈ < 1s batch updates < 1ms iterations < 100ms interactive queries

5 Outline Revisiting dataflow How to achieve low latency Evaluation

6 Dataflow Stage Connector

7 Dataflow: parallelism BC Vertex Edge

8 Dataflow: iteration

9 BatchingStreaming vs.  Requires coordination Supports aggregation No coordination needed  Aggregation is difficult (synchronous)(asynchronous)

10 Batch iteration 

11 Streaming iteration  

12 Timely dataflow Supports asynchronous and fine-grained synchronous execution   – timestamp

13 How to achieve low latency Programming model Distributed progress tracking protocol System performance engineering

14 Programming model BCD C.O PERATION (x, y, z) C.O N C ALLBACK (u, v) 2×2× 2×2×

15 Messages BCD B.S END B Y (edge, message, time) C.O N R ECV (edge, message, time)  Messages are delivered asynchronously

16 Notifications BCD D.N OTIFY A T (time) D.O N N OTIFY (time) Notifications support batching C.S END B Y (_, _, time)  No more messages at time or earlier D.O N R ECV (_, _, time)

17 Programming frameworks Timely dataflow API Distributed runtime Frameworks LINQ GraphLINQ Differential dataflow BSP (Pregel) BLOOM AllReduce input.SelectMany(x => x.Split()).Where(x => x.StartsWith("#")).Count(x => x);

18 How to achieve low latency Programming model Asynchronous and fine-grained synchronous execution Distributed progress tracking protocol System performance engineering

19 How to achieve low latency Programming model Asynchronous and fine-grained synchronous execution Distributed progress tracking protocol System performance engineering

20 Progress tracking E.N OTIFY A T (t) ABCDE C.O N R ECV (_, _, t) C.S END B Y (_, _, t ʹ ) t ʹ ≥ t Epoch t is complete

21 Progress tracking A BC D E Problem: C depends on its own output C.N OTIFY A T (t)

22 A BC D E C.N OTIFY A T ((1, 6)) D.S END B Y (1, 6) A.S END B Y (_, _, 1) E.N OTIFY A T (?) B.S END B Y (_, _, (1, 7)) F Advances timestamp Advances loop counter E.N OTIFY A T (1) Solution: structured timestamps in loops C.N OTIFY A T (t)

23 (1, 6) Graph structure leads to an order on events A BC D E (1, 5) (1, 6) 1 ⊤ F

24 Graph structure leads to an order on events A BC D E (1, 5) (1, 6) 1 ⊤ F O N N OTIFY (t) is called after all calls to O N R ECV (_, _, t) 1.Maintain the set of outstanding events 2.Sort events by could-result-in (partial) order 3.Deliver notifications in the frontier of the set

25 D.O N R ECV (_, _, (1, 5)) C.S END B Y (_, _, (1, 5)) E.O N N OTIFY (1) E.N OTIFY A T (1) Optimizations make doing this practical

26 How to achieve low latency Programming model Asynchronous and fine-grained synchronous execution Distributed progress tracking protocol Enables processes to deliver notifications promptly System performance engineering

27 How to achieve low latency Programming model Asynchronous and fine-grained synchronous execution Distributed progress tracking protocol Enables processes to deliver notifications promptly System performance engineering

28 Performance engineering Microstragglers are the primary challenge Garbage collectionO(1–10 s) TCP timeoutsO(10–100 ms) Data structure contentionO(1 ms) For detail on how we handled these, see paper (Sec. 3)

29 How to achieve low latency Programming model Asynchronous and fine-grained synchronous execution Distributed progress tracking protocol Enables processes to deliver notifications promptly System performance engineering Mitigates the effect of microstragglers

30 Outline Revisiting dataflow How to achieve low latency Evaluation

31 System design Limitation: Fault tolerance via checkpointing/logging (see paper) SSSS Progress tracker SSSS Data Control 64  8-core 2.1 GHz AMD Opteron 16 GB RAM per server Gigabit Ethernet

32 Iteration latency 64  8-core 2.1 GHz AMD Opteron 16 GB RAM per server Gigabit Ethernet Median: 750 μs 95 th percentile: 2.2 ms

33 Timely dataflow API Distributed runtime Frameworks LINQ GraphLINQ Differential dataflow BSP (Pregel) BLOOM Applications Word count Iterative machine learning PageRank Interactive graph analysis AllReduce

34 PageRank Pregel (Naiad) GraphLINQ GAS (Naiad) GAS (PowerGraph) Twitter graph 42 million nodes 1.5 billion edges 64  8-core 2.1 GHz AMD Opteron 16 GB RAM per server Gigabit Ethernet

35 32K tweets/s 10 queries/s Interactive graph analysis In ⋈ z? ⋈ max ⋈

36 Query latency 32  8-core 2.1 GHz AMD Opteron 16 GB RAM per server Gigabit Ethernet Max: 140 ms 99 th percentile: 70 ms Median: 5.2 ms

37 Conclusions Low-latency distributed computation enables Naiad to: achieve the performance of specialized frameworks provide the flexibility of a generic framework Now available for download: The timely dataflow API enables parallel innovation

38 For more information Visit the project website and blog Now available for download:

39 Naiad   Now available for download:


Download ppt "Naiad: A Timely Dataflow System Derek G. Murray Michael Isard Frank McSherry Paul Barham Rebecca Isaacs Martín Abadi Microsoft Research 1."

Similar presentations


Ads by Google