1. Orchestration by Approximation Mapping Stream Programs onto Multicore Architectures S. M. Farhad (University of Sydney) Joint work with Yousun Ko Bernd Burgstaller Bernhard Scholz

2. 2 Outline Motivation Research question Contributions Summary 2

3. 3 Cores are the New Gates # cores/chip ( Shekhar Borkar, Intel) Courtesy: Scott’08 C/C++/Java CUDA X10 Peakstream Fortress Accelerator Ct C T M Rstream Rapidmind Stream Programming 3

4. 4 Stream Programming Paradigm Programs expressed as stream graphs  Streams: Sequence of data elements  Actor: Functions applied to streams 4 Actor/Filter Streams

5. 5 StreamIt Language Basic and hierarchical structures Each construct has single input/output stream parallel computation may be any StreamIt language construct joiner splitter pipeline feedback loop joiner splitter splitjoin filter 5

6. 6 Outline Motivation Research question Contributions Summary 6

7. 7 How to Orchestrate a Stream Graph? Mapping actors Eliminate bottlenecks (aka. hot actors) 7

8. Mapping Actors 8 B 60 C D 5 5 A Core 1Core 2Core 3 5 A D 5 B 60 C Load=10Load=60 Make span = 60, Speedup = 130/60 = 2.17 Ideally speedup = 3 Actors B and C are the bottlenecks

9. Bottlenecks Elimination 9 B 60 C D 5 5 A D 15 A B_1 60 6 s1 6 j1 B_2 60 B_3 60 C_1 60 6 s2 6 j2 C_2 60 C_3 60 Hot actor duplication Core 1 Core 2 Core 3 15 A s1 6 B_1 60 C_1 60 B_2 60 j1 6 s2 6 C_2 60 j2 6 B_3 60 C_3 60 D 15 Load = 141Load = 138Load = 135 Make span = 141, Speedup = 130x3/141 = 2.77 28% increased speedup

10. 10 Orchestration of Stream Program Contd. Current state of the art  Integer Linear Programming Intractable  Heuristics Unknown performance How to find a fast and good solution?  Approximation algorithms that have Polynomial runtime Quality bound for solution 10

11. 11 Outline Motivation Research question Contributions Summary 11

12. 12 Data Transfer Model Arrival rate depends on the data rate of the actors (maximize) Data transfer model forms a system of sim. functional linear equation Compute a closed form of the output data rate We also consider a processor utilization function for each actor 12 ABC 1511 z

13. 13 Bottleneck Analysis The arrival rate is limited by  Processor capacity of the cores  Memory bandwidth A quantitative analysis determines  An upper bound of the arrival rate imposed by an actor  An upper bound of the arrival rate imposed by the parallel system Hot actor  Upper bound (actor) < upper bound (system) 13

14. 14 Approximation of Actor Allocation Problem The actor allocation problem (AAP) is NP-hard For a fixed arrival rate, the AAP reduces to standard bin-packing problem (closed form) There exist approximation algorithms for bin- packing  Polynomial running time  Solution quality is bounded

15. 15 Summary (ASPLOS 2011) Novel data transfer model A simple quantitative analysis to detect and eliminate bottlenecks A novel 2-approximation algorithm for deploying stream graphs on multicore platforms Results are within 5% of the optimal solution Achieves a geometric mean speedup of 6.95x for 8 processors over single processor execution 15

16. 16 Related Works [1] Static Scheduling of SDF Programs for DSP [IEEE ’87] [2] StreamIt: A language for streaming applications [Springer ‘02] [3] Phased Scheduling of Stream Programs [LCTES ’03] [4] Exploiting Coarse Grained Task, Data, and Pipeline Parallelism in Stream Programs [ASPLOS ‘06] [5] Orchestrating the Execution of Stream Programs on Cell [PLDI ’08] [6] Software Pipelined Execution of Stream Programs on GPUs [IEEE ‘09] [7] Synergistic Execution of Stream Programs on Multicores with Accelerators [IEEE ‘09] [8] An empirical characterization of stream programs and its implications for language and compiler design [PACT ’10] 16

17. Questions?

18. 18 Focus of Our Work Stream Graph Scheduling Stream Graph Partitioning Layout on Target Architecture Communication Scheduling Linear Functional Equation Solver Actor Allocation on Processors Bottleneck Resolver StreamIt Compiler Phases

19. 19 Actor Allocation Constraint 19 100% Actors with their utilizations Each core has 100% utilization 1 3 4 5 n 2 1 n 5 2 3

20. 20 Binary Search 20 0ub(z) 1.0 Solution space rightleftmid Allocation possible?

21. 21 Binary Search 21 0ub(z) 1.0 rightleft Allocation possible? mid 21

22. 22 Binary Search 22 0ub(z) 1.0 rightleft Allocation possible? mid 22

23. 23 Actor Allocation of Bottleneck Free Program 23 D 5 5 A B_1 20 2 s1 B_2 20 B_3 20 C_1 20 2 j1 C_2 20 C_3 20 Core 1 5 A B_1 20 C_1 20 Core 2 s1 2 B_2 20 j1 2 C_2 20 Core 3 B_3 20 C_3 20 D 5 Load = 45Load = 44Load = 45 Make span = 45, Speedup = 130/45 = 2.89 Mapping Efficient Bottleneck Resolving

24. 24 Experiments Our method implemented as an extension of StreamIt compiler We compare to ILP based method [Scott 08] (solved with CPLEX) Hardware Setup  2.33GHz dual quad-core Intel Xeon processors 16GB memory Linux kernel version 2.6.23  Profiler uses the x86-64’s hardware cycle counters 24

25. 25 Experiments Contd. Experimental Process  Profiling  Computing closed form  Resolve bottlenecks  Compute the mapping  Compute the layout scheduling  Invoke the StreamIt back end  Finally we measure the performance 25

26. 26 Experiments Contd. Benchmark ActorsStateful DCT2218 FMRadio6723 TDE5527 FFT2614 MergeSort312 FilterBankNew5334 RadixSort132 EqualizerProgram6522 BitonicSort4522 DES375180 MPEG397 MatrixMult522 26

27. 27 Experimental Results for 2 – 4 Processors BenchmarkProc# ILP Time (Optimal)(s) DCT 20.27 31585.69 42285.01 FMRadio 20.08 33.22 41.29 TDE 20.09 30.17 4274.69 FFT 20.46 344694.25 4249240.09 27 Our method’s run time: <1s BenchmarkProc# ILP Time (Optimal)(s) Equalizer 20.06 35.29 457553.83 BitonicSort 20.3 33.06 416371.99 DES 20.51 32.73 411.24 MPEG 20.12 31.37 40.44

28. 28 Experimental Results for 2 – 4 Processors BenchmarkProc# Arrival rate ratio (Appx/Optimal) Apx. Arrival Rate (MBps) DCT 20.9942.56 30.9762.46 40.9682.57 FMRadio 21.002.69 31.004.00 40.995.34 TDE 21.0013.31 31.0019.80 41.0028.81 FFT 20.9843.91 30.9846.85 40.9595.50 28 BenchmarkProc# Arrival rate ratio (Appx/Optimal) Apx. Arrival Rate (MBps) Equalizer 21.000.56 31.000.83 41.001.59 BitonicSort 21.003.16 31.004.73 41.0010.14 DES 21.000.12 31.000.18 41.000.24 MPEG 21.0036.59 30.9954.68 41.0073.22

29. 29 Speedup Results 29

30. 30 Summary Approximation algorithm for solving actor allocation problem Data rate transfer model that resolves bottlenecks We separate the bottleneck elimination from the actor allocation We implemented our approach and compared with an optimal approach Optimal approach has unpredictable time Our approach has negligible time for all benchmarks Quality of our approach is at most 5% off the optimum For up to 8 processors we achieve a geometric mean speedup of 6.95x over single processor execution 30

31. Questions?