1. Pipelined Computations P0P1P2P3P5

2. Pipelined Computations a s in s out a[0] a s in s out a[1] a s in s out a[2] a s in s out a[3] a s in s out a[4] for(i=0 ; i<n ; i++) sum = sum + a[i]; sum = sum + a[0]; sum = sum + a[1]; sum = sum + a[2]; sum = sum + a[3]; sum = sum + a[4];

3. Pipelined Computations f0f0 f in f out f1f1 f in f out f2f2 f in f out f3f3 f in f out f4f4 f in f out Signal without f 0 Signal without f 1 Signal without f 2 Signal without f 3

4. Pipelined Computations 1.If more than one instance of the complete problem is to be executed. 2.If a series of data items must be processed, each requiring multiple operations 3.If information to start the next process can be passed forward before the process has completed all its internal operations

5. Pipelined Computations P5 P4 P3 P2 P1 P0 12 12 1 3 2 1 3 4 2 1 3 4 5 2 1 3 4 5 6 2 3 4 5 6 7 3 4 5 6 7 4 5 6 7 5 6 7 mp-1 The average number of cycles is (m+p-1) cycles.

6. Pipelined Computations P4P3P2P1P0P5 P4P3P2P1P0P5 P4P3P2P1P0P5 P4P3P2P1P0P5 P4P3P2P1P0P5 I0 I1 I2 I3 I4 … time

7. Pipelined Computations P0P1P2P3P4P5P6P7P8P9 d9d8d7d6d5d4d3d2d1d0d9d8d7d6d5d4d3d2d1d0 P8 P7 P6 P5 P4 P3 P2 P1 P0 123456789 123456789 123456789 123456789 123456789 123456789 123456789 123456789 123456789 p-1n

8. Pipelined Computations P0 P1 P2 P3 P4 P5 P0 P1 P2 P3 P4 P5

9. Pipelined Computations P0P1P2P3 Processor 0 P4P5P6P7 Processor 1 P8P9P10P11 Processor 2

10. Pipelined Computations Host computer An ideal interconnection structure is a line or ring structure.

11. Adding numbers P0P1P2P3P4 recv(&accumulation, Pi-1); accumulation = accumulation + number; send(&accumulation, Pi+1);

12. Adding numbers P 0 : send(&number, P 1 ); P n-1 : recv(&number, P n-2 ); accumulation = accumulation + number; send(&number, P 0 ); if( process > 0){ recv(&accumulation, Pi-1); accumulation = accumulation + number; } if( process < n-1) send(&accumulation, Pi+1);

13. Adding numbers P0P1P2P4 d n-1 …..d 2 d 1 d 0 sum

14. Adding numbers P0P1P2Pn-1 d n-1 …..d 2 d 1 d 0 sum …..

15. Adding numbers Analysis : t total =(time for one pipeline cycle)(number of cycles) t total =(t comp +t comm )(m+p-1) t a = t total /m Single Instance of Problem : t comp = 1 t comp = 2(t startup +t data ) t comp = (2(t startup +t data )+1)n

16. Adding numbers Multiple Instance of Problem: t comp = (2(t startup +t data )+1)(m+n-1) t a = t total /m=2(t startup +t data )+1

17. Adding numbers Data partitioning with Multiple Instances of problem t comp = d t comp = 2(t startup +t data ) t comp = (2(t startup +t data )+d)(m+n/d-1)

18. Sorting Numbers P0 P1 P2 P3 P4 5 5 52 52 531 542 5431 5432 54321 4,3,1,2,5 4,3,1,2 4,3,1 4,3 4 2 1 13 42 31 2 1 1 2 3 4 5 6 7 8 9 10

19. Sorting Numbers recv(&number, Pi-1); if(number > x) { send(&x, Pi+1); x = number; } else send(&number, Pi+1); right_procno = n-i-1; recv(&x, Pi-1); for(j=0 ; j<right_procno ; j++) recv(&number, Pi-1); if(&number > x){ send(&x, Pi+1); x = number; } else send(&number, Pi+1); }

20. Sorting Numbers x max compare x n-1..x 1 x 0 x max compare x max compare P0 P1 P2 smaller numbers

21. Sorting Numbers P0P1P2 Pn-1 d n-1 …..d 2 d 1 d 0 Master right_procno = n-i-1; recv(&x, Pi-1); for(j=0 ; j<right_procno ; j++) recv(&number, Pi-1); if(&number > x){ send(&x, Pi+1); x = number; } else send(&number, Pi+1); } send(&number, Pi-1); for(j=0 ; j<right_procno ; j++){ recv(&x, Pi+1); send(&x, Pi-1); }

22. Sorting Numbers analysis : Sorting phase Returning sorted numbers 2n-1 n P4 P3 P2 P1 P0

23. Primer number Generation 2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20 To find the primes up to n, it is only necessary to start at numbers up to

24. Primer number Generation Analysis for(i=2 ; i<n ; i++) prime[i] = 1; for(i=2 ; i<=sqrt_n ; i++) if(prime[i] == 1) for(j=i+1 ; i<n ; j=j+i) prime[j] = 0; Sequential code:

25. Primer number Generation Parallel Code recv(&x, Pi-1) recv(&number, Pi-1); if((number%x) !=0) send(&number, Pi+1); p1p1 compare x n-1..x 1 x 0 p2p2 compare p3p3 P0 P1 P2 Not multiple of 1 st prime number recv(&x, Pi-1); for(i=0 ; i<n ; i++){ recv(&number, Pi-1); if(number == terminator) break; if((number%x) !=0 ) send(&number, Pi-1); }

26. Solving a System of Linear Equations Upper-triangular

27. Solving a System of Linear Equations Sequential Code x[0] = b[0] / a[0][0]; for(i=1 ; i<n ; i++){ sum = 0; for(j=0 ; j<i ; j++){ sum = sum + a[i][j]*x[j]; x[i]=(b[i]-sum)/a[i][i]; }

28. Solving a System of Linear Equations Compute x0Compute x1Compute x2Compute x3 X0 X1 X2 X3 X0 X1 X2 x0 x1 x0

29. Solving a System of Linear Equations Parallel Code for(j=0 ; j<i ; j++){ recv(&x[j], Pi-1); send(&x[j], Pi+1); } sum=0; for(j=0 ; j<i ; j++) sum=sum+a[i][j]*x[j]; x[i] = (b[I]-sum)/a[i][i]; Send(&x[i], Pi+1);

30. Solving a System of Linear Equations Parallel Code : P i sum = 0; for(j=0 ; j<i ; j++){ recv(&x[j], Pi-1); send(&x[j], Pi+1); sum = sum + a[i][j]*x[j]; } x[i]=(b[i]-sum)/a[i][i]; send(&x[i], Pi+1);

31. divide send(x0) end recv(x0) send(x0) multiply/add divide/subtract send(x1) end recv(x0) send(x0) multiply/add recv(x1) send(x1) multiply/add divide/subtract send(x2) end recv(x0) send(x0) multiply/add recv(x1) send(x1) multiply/add divide/subtract recv(x2) send(x2) multiply/add divide/subtract send(x3) end recv(x0) send(x0) multiply/add recv(x1) send(x1) multiply/add divide/subtract recv(x2) send(x2) multiply/add divide/subtract recv(x3) send(x3) multiply/add divide/subtract send(x4) end time P0 P1 P2 P3 P4 Solving a System of Linear Equations

32. P5 P4 P3 P2 P1 P0 First value passed onward final computed value