Computer Science 320 Load Balancing

Behavior of Parallel Program Why do 3 threads take longer than two?

Get running times of each thread Ideally, each one should run for T / K Only when K = 2 is this the case The threads assigned to the middle rows of the matrix always do more work

The Cause of the Problem Each thread takes same # of iterations Thread 1 takes more iterations. Why?

Load Balance The extent to which each processor (thread) does the same amount of work If the work is the same, we have a balanced load If not, we have an unbalanced load

Quantifying Load Balance B = T p (K) / (T p (1) / K) = K * T p (K) / T p (1)

Achieving a Balanced Load If some threads do more work on some rows, then the rows cannot be evenly divided among them Some thread will get more rows, ideally, the rows with the fewest points in the Mandelbrot set

PJ resources for Load Balancing Can divide iterations of a parallel for loop unevenly among the threads, by setting the loop’s schedule Different schedules can be set in the program code or via the command line

Fixed Schedule By default, each thread gets a chunk of loop iterations of the same size 100 iterations, 4 threads

Dynamic Schedule Chunk sizes are the same, there can be many more chunks than there are threads When a thread’s run method completes, it’s called again on a different chunk Default chunk size is 1 iteration, but it can be made larger 100 iterations, chunk size 5

Guided Schedule Chunk sizes decrease in size, by a factor of 2 * K, with a default minimum size of 1 When a thread’s run method completes, it’s called again on a different chunk K = 1 K = iterations

On Command Line or in Code java –Dpj.schedule=guided new IntegerForLoop(){ public IntegerSchedule schedule(){ return IntegerSchedule.guided(); }... }