1. CS 420 Design of Algorithms Parallel Algorithm Design

2. Graph Partition and Mapping Remember: Minimize idle processes Minimize process interaction Minimize interprocess task interaction

3. Graph partitioning Consider finite element mesh modeling and computation See example of regular mesh For regular structured meshes this is pretty straight forward

4. Graph Partitioning But consider an irregular mesh … A water quality model of Lake Superior A task processes a mesh point … must factor in neighbor mesh points How would you map this to 8 processes… to.. Minimize interaction Minimize idling (load balancing)

5. Graph partitioning Consider a sparse Matrix-Vector multiplication problem… Simple partitioning See GK Fg. 3.38 Consider a task interaction graph Partition based on task interaction graph Which has the lowest number of interactions?

6. Dynamic Mapping Centralized vs Distributed Mapping

7. Centralized Dynamic Mapping Master/slave model Master process (or subset of processes) When worker process idle – notifies master process and requests new work (task) In response master process assigns new task When no more work to assign = job done

8. Centralized dynamic mapping Master/slave model Consider the job of sorting the columns of a matrix Master assigns a task (a row to sort) to each worker process The runtime of each task depends on the order of input column If worker process finishes early- it would be idle Idle worker process checks with Master process for new task (another column to sort.

9. Centralized dynamic mapping Master/slave scheme Problem: if tasks are small and T(t) is small task requests may swamp master task – Task requests get queued = bottleneck Can assign tasks in larger pieces (chunk scheduling) Chunk sizing and mapping is somewhat of an art

10. Distributed dynamic mapping Work is assigned to peer processes Peers can exchange tasks as needed or as processes are available.