1. 1 Friday, September 29, 2006 If all you have is a hammer, then everything looks like a nail. -Anonymous

2. 2 §Domain Decomposition 1. Divide data in approx. equal parts 2. Partition the computation §Functional Decomposition 1. Divide the computation into disjoint tasks 2. Determine data requirements of these tasks

3. 3 §Domain Decomposition 1. Divide data in approx. equal parts 2. Partition the computation §Functional Decomposition 1. Divide the computation into disjoint tasks 2. Determine data requirements of these tasks l If data is disjoint then partition is complete l If there is significant overlap  consider domain decomposition.

4. 4 Task dependency graph?

5. 5 §Typically maximum degree of concurrency is less than the total number of tasks.

6. 6 §Degree of concurrency depends on shape of task dependency graph.

7. 7

8. 8 Mapping §Maximize use of concurrency. §Task dependencies and interactions are important in selection of good mapping. §Minimize completion time by making sure that the processes on critical path execute as soon as they are ready.

9. 9 Mapping §Maximize concurrency and minimize interaction among processors l Place tasks that are able to execute independently on different processors to increase concurrency. l Place tasks that communicate frequently on same processor to increase locality.

10. 10 Mapping Cannot use more than 4 processors. Why?

11. 11 Mapping Prevent inter-task interaction from becoming inter-process interaction

12. 12 Agglomeration

13. 13 Data partitioning: Block distribution Higher dimensional distributions may help reduce the amount of shared data that needs to be accessed

14. 14 Data partitioning: Block distribution Higher dimensional distributions may help reduce the amount of shared data that needs to be accessed. n 2 /p +n 2 vs. 2n 2 /√p

15. 15 Sum N numbers §N numbers are distributed among N tasks Centralized algorithm

16. 16 Sum N numbers §N numbers are distributed among N tasks Distributing computation

17. 17 Recursive Decomposition §Divide and conquer §Set of independent sub-problems

18. 18 Recursive Decomposition §Sum N numbers. How many steps required?

19. 19

20. 20

21. 21 Hybrid decomposition §Possible to combine different techniques §Finding minimum of a large set of numbers by purely recursive decomposition is not efficient.

22. 22 Hybrid decomposition

23. 23 Exploratory Decomposition

24. 24 Unfinished tasks can be terminated once solution is found

25. 25

26. 26 §Read: Speculative Decomposition

27. 27 Communications §Most parallel problems need to communicate data between different tasks.

28. 28 Embarrassingly Parallel Applications §No communication between tasks. §One end of spectrum of parallelization §Examples?

29. 29 Factors involving communication §Machine cycles and resources that could be used for computation are instead used to package and transmit data. §Sending many small messages can cause latency to dominate communication overheads. §Package small messages into a larger message results in increased bandwidth.

30. 30 Factors involving communication §Synchronous communications. §Asynchronous communications. §Interleaving computation with communication.

31. 31 npoints = 10000 circle_count = 0 do j = 1,npoints generate 2 random numbers between 0 and 1 xcoordinate = random1 ; ycoordinate = random2 ; if (xcoordinate, ycoordinate) inside circle then circle_count = circle_count + 1 end do PI = 4.0*circle_count/npoints

32. 32