1. Computer Science and Engineering Parallel and Distributed Processing CSE 8380 March 01, 2005 Session 14

2. Computer Science and Engineering Scheduling Introduction Model Program tasks Machine Schedule Execution and communication time Problem Complexity

3. Computer Science and Engineering Introduction to Scheduling This problem has been described in a number of different ways in different fields Classical problem of job sequencing in production management has influenced most of the solutions Set of resources and set of consumers

4. Computer Science and Engineering Scheduling System ConsumersResourcesScheduler Policy

5. Computer Science and Engineering Partitioner Grains of Sequential Code Parallel/Distributed System Parallel Program Tasks Scheduler Schedule Processors Time Program Tasks Sequential Program Explicit Approach Implicit Approach Dependence Analyzer Ideal Parallelism Scheduling Parallel Tasks

6. Computer Science and Engineering Program Tasks (T, <, D, A) T  set of tasks <  partial order on T D  Communication Data A  amount of computation

7. Computer Science and Engineering Task Graph A 10 D 15 E 10 F 20 B 15 C 10 G 15 H I 30 5 5 87 5 55 10 5 4 54 20

8. Computer Science and Engineering Machine m heterogeneous processors Connected via an arbitrary interconnection network (network graph) Associated with each processor P i is its speed S i Associated with each edge (i,j) is the transfer rate R ij

9. Computer Science and Engineering Examples Machines Linear Array Ring Mesh 63 12 54 Fully Connected

10. Computer Science and Engineering Task Schedule Gantt Chart Mapping (f) of tasks to a processing element and a starting time Formally: f: T  {1,2,3, …, m} x [0  i nfinity  f(v) = (i,t)  task v is scheduled to be processed by processor i starting at time t

11. Computer Science and Engineering Gantt Chart

12. Computer Science and Engineering Gantt Chart with Communication

13. Computer Science and Engineering Execution and Communication Times If task t i is executed on p j Execution time = A i /S j The communication delay between t i and t j, when executed on adjacent processing elements p k and p l is D ij /R kl

14. Computer Science and Engineering Complexity Computationally intractable in general Small number of polynomial optimal algorithms in restricted cases A large number of heuristics in more general cases schedulescheduler Quality of the schedule vs. Quality of the scheduler

15. Computer Science and Engineering Scheduling Task Graphs without considering communication Polynomial-Time Optimal Algorithms in the following cases: 1. Task graph is in-forest or out-forest 2. Task graph is an interval order 3. Only two processors

16. Computer Science and Engineering Assumptions A task graph consisting of n tasks A distributed system made up of m processors The execution time of each task is one unit of time Communication between any pair of tasks is zero The goal is to find an optimal schedule, which minimizes the completion time

17. Computer Science and Engineering List Scheduling All three algorithms belong to the list scheduling class. Each task is assigned a priority, and a list of tasks is constructed in a decreasing priority order. A task becomes ready for execution when its immediate predecessors in the task graph have already been executed or if it does not have any predecessors.

18. Computer Science and Engineering Scheduling inforest/outforest task graphs 1. The level of each node in the task graph is calculated as given above and used as each node’s priority 2. Whenever a processor becomes available, assign it the unexecuted ready task with the highest priority (How about opposing forest?)

19. Computer Science and Engineering Example