1. Load Balancing : The Goal Given a collection of tasks comprising a computation and a set of computers on which these tasks may be executed, find the mapping of tasks to computers that results in each computer having an approximate equal amount of work

2. Static Vs. Dynamic Static: If the computation time and behavior of a given task can be determined a priori, task mapping can be performed before beginning of computation. Dynamic: If the workload for a task can change over the course of a computation and cannot be estimated beforehand, task mapping must change during the computation.

3. Approach to Dynamic Load Balancing Load Evaluation Profitability Determination Work Transfer Vector Calculation Task Selection Task Migration

4. Load Evaluation Application-based approach System-based approach Hybrid approach

5. Profitability Determination Load balance of a computation is the ratio of the average computer load to the maximum computer load Load of computation can be measured locally or globally.

6. Work Transfer Calculation Calculate how much work should be transferred from computer to another –Hierarchical Balancing Method –Generalized Dimensional Exchange –Diffusive Techniques Work transferring while preserving communication locality

7. Hierarchical Balancing (HB) Global and recursive approach –A set of computers is divided in half –The load is calculated for each partition –Transfer vector is derived so that those partitions have roughly the same load per computer. –Each partition is self divided and balanced recursively, taken into account transfers calculated at higher level

8. Dimensional Exchange (DE) For a network of max degree = Nmax, Computers exchange with its neighbor in each dimension, times their load difference. The process is repeated until balanced state is reach. for each neighbor j in N i

9. Diffusive Techniques Based on the solution of diffusion equation Uses implicit differencing scheme to solve the heat equation on a hyper-cube

10. Task Selection (1) Determined which tasks should be moved to meet the calculated transfer vectors –Move tasks from one computer to another –Exchange tasks between two computers Basic idea is to find the subset of sum (0-1 Knapsack) Rules: –Try to avoid moving large numbers of tasks or large quantities of data. –Try best to preserve communication locality.

11. Task Selection (2) Each task, i has a load L i a cost C i. For a given transfer, we wish to find the minimum-cost set of tasks whose exchange achieves that transfer. The selection algorithm cannot satisfy a transfer vector in a single attempt. Multiple passes will be required. To avoid store-and-forward situation, the transfer of tokens that contain load/cost information should be performed until task selection is complete.

12. Task Migration (1) In addition to task selection, load balancing system should also provides mechanisms to actually move the tasks from one computer to another. Task movement must preserve the task’s state and pending communication.

13. Task Migration (2) Studies show that reducing the size of the task transfer is more important than reducing the number of tasks transferred. Task transfer should not disrupt the communication locality. If communication costs are significantly increased, it may be better not to load balance at all.

14. Load Balancing System (SCPLib) Library Routines Comm. List Application Routines State userLibrary

15. Conclusion Dynamic load balancing is more flexible than static load balancing and can support irregular/unpredictable problems. Approaches to dynamic load balancing includes: load evaluation, profit determination, work transfer calculation, task selection and migration. Load balancing should be applied only if it does not disrupt the communication locality.