Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Dataflow Programming Language and Its Compiler for Streaming Systems Haitao Wei, Stephane Zuckerman, Xiaoming Li, and Guang Gao University of Delaware.

Similar presentations


Presentation on theme: "A Dataflow Programming Language and Its Compiler for Streaming Systems Haitao Wei, Stephane Zuckerman, Xiaoming Li, and Guang Gao University of Delaware."— Presentation transcript:

1 A Dataflow Programming Language and Its Compiler for Streaming Systems Haitao Wei, Stephane Zuckerman, Xiaoming Li, and Guang Gao University of Delaware

2 Why a new dataflow programming language? 2

3 3

4 4

5 5  Arbitrary dataflow graph with multiple input/output operators Beyond static mapping True asynchronous processing  Primitive and compositional coding Code reuse: using subgraph template Easy to construct dataflow graph: using explicit variable/name passing to connect the computation node  Efficiency under dynamic program configurations Automatic software pipelining Optimized resource allocation and mapping

6 COStream Programming Language  Multiple I/O Operators VS Single I/O Operator 6 Fork-join style dataflow graph in Streamit with single input/output and split-join structure Arbitrary Dataflow graph in COSTream with multiple Input/Output Operator and stream Split Join

7 COStream Programming Language  composite Connect operators to construct a dataflow subgraph Can be instantiated to reuse the code to explore the parallelism

8 COStream Programming Language 8 composite Main(){ graph stream S=A(){… } stream T=B(){…} stream P=C(T,S){…} } A B C  Stream variable and dataflow graph construct Split AB Join C StreamIt: Introduce useless split-join nodes which makes programming hard and brings the overhead Costream: Easy to construct and make it simple

9 Example: Moving Average 9 Average … N 1 Source Sink composite Main(){ Int N=10; graph stream S=Source(){ int x; init{x=0;} work{ S[0].x=x; x++; } //initialize an instance of composite MovAve stream P=MovAver(S)(N); Sink(P){ work{ print(P[0].x); } }} composite MovAver(output O, input rawIn){ param int N; graph stream O=Aver(rawIn){ float w[N]; init{ for(i=0;i

10 Example: Moving Average 10 Average … N 1 Source Sink composite Main(){ Int N=10; graph stream S=Source(){ int x; init{x=0;} work{ S[0].x=x; x++; } //initialize an instance of composite MovAve stream P=MovAver(S)(N); Sink(P){ work{ print(P[0].x); } }} composite MovAver(output O, input rawIn){ param int N; graph stream O=Aver(rawIn){ float w[N]; init{ for(i=0;i

11 Example: Moving Average 11 Average … N 1 Source Sink composite Main(){ Int N=10; graph stream S=Source(){ int x; init{x=0;} work{ S[0].x=x; x++; } //initialize an instance of composite MovAve stream P=MovAver(S)(N); Sink(P){ work{ print(P[0].x); } }} composite MovAver(output O, input rawIn){ param int N; graph stream O=Aver(rawIn){ float w[N]; init{ for(i=0;i

12 Example: Moving Average 12 Average … N 1 Source Sink composite Main(){ Int N=10; graph stream S=Source(){ int x; init{x=0;} work{ S[0].x=x; x++; } //initialize an instance of composite MovAve stream P=MovAver(S)(N); Sink(P){ work{ print(P[0].x); } }} composite MovAver(output O, input rawIn){ param int N; graph stream O=Aver(rawIn){ float w[N]; init{ for(i=0;i

13 Example: Moving Average 13 Average … N 1 Source Sink composite Main(){ Int N=10; graph stream S=Source(){ int x; init{x=0;} work{ S[0].x=x; x++; } //initialize an instance of composite MovAve stream P=MovAver(S)(N); Sink(P){ work{ print(P[0].x); } }} composite MovAver(output O, input rawIn){ param int N; graph stream O=Aver(rawIn){ float w[N]; init{ for(i=0;i

14 Compiler Framework 14 Front-end Lexer, parser and construct AST Front-end Lexer, parser and construct AST MIR Construct stream graph from AST MIR Construct stream graph from AST Symbol Execute Initial && Periodic schedule of SG Symbol Execute Initial && Periodic schedule of SG Optimization Parallel exploring Buffer allocation Synchronization Optimization Parallel exploring Buffer allocation Synchronization Code Generation Runtime System Runtime System Low-level c/c++ compiler Multi/Many- core AC B D E Synchronization Data Flow COStream program

15 Software Pipelining 15 A0 21 A0 21 A1 21 A1 21 B09B09 B09B09 B19B19 B19B19 B2 9 B2 9 Core1Core2 Core3 Step1:Task Assignment

16 Software Pipelining Stage1 Stage0 A0 21 A0 21 A1 21 A1 21 B09B09 B09B09 B19B19 B19B19 B2 9 B2 9 Step2:Stage Assignment

17 Software Pipelining 17 Time B0B0 B1 B2 Core1Core2Core3 A0A1 B0B0 B1 B2 A0A1 A0A1 … Software Pipelining Barrier

18

19

20 Ongoing Research  Compilation Software Pipelining within codelets  Backend Currently using pthread Will use Fresh Breeze runtime 20

21 Ongoing Research (cont’d)  Runtime Currently re-optimize for every new configuration Graceful transition between configurations 21

22 Transition between configurations 22

23 Transition between configurations 23

24 Transition between configurations 24

25 Transition between configurations 25

26 Acknowledgements AFOSR FA NSF CCF , CCF and OCI


Download ppt "A Dataflow Programming Language and Its Compiler for Streaming Systems Haitao Wei, Stephane Zuckerman, Xiaoming Li, and Guang Gao University of Delaware."

Similar presentations


Ads by Google