Pitfalls: Time Dependent Behaviors CS433 Spring 2001 Laxmikant Kale

2 Plan for this week Monday: Pitfalls: time dependent behaviors Wednesday: loop/data parallelism: openMP, Fri: Loop prallelism on distributed memory machines: –High Performance Fortran MP1 is coming out tomorrow Project list will be out tomorrow –Projects and info will be added to the list as and when necessary Next week: parallel application classes

3 What makes parallel programming Many things, but one of them is: time dependent behavior –Other things include performance! This issue makes it harder to get a parallel program running correctly Affects all 3 models: –MPI –Pthreads/ SAS –Charm++

4 SAS model Locks and Deadlocks Race conditions

5 Deadlocks Defined as a “circular wait” Example: –Two threads, A and B, two locks (M and N) –Thread A locks lock M, and tries to lock N –Thread B locks N, and tries to lock M –They may both do there work most of the times, –but once in a while the program locks up… A: repeat compute(); lock(M); lock(N); critical(); unlock both B: repeat compute(); lock(N); lock(M); critical(); unlock both

6 Deadlock solution strategy Lock only one lock at a time : –impractical Order the locks –Always request the locks in a particular order, and release them in the reverse order –This is sometimes hard to do. The statements may not be next to each other as nicely as in this example You lock what you need to as you go along in the code… Break the deadlock when it happens –This strategy is used in databases and distributed computing –Not much in parallel computing

7 Race conditions The result, output or behavior of the parallel program differes from run to run: –Example: queue of tasks Each process: repeat { task = dequeue(); print threadnum, tasknum} Even if queue is protected by lock, the output pairs of numbers may be different

8 Race conditions: Kinds of race conditions and how handle them –Benign (as in previous example): ignore –Priority Queue in a Search program The set of explored nodes may differ from run to run –Asynchronous algorithms: allows race conditions, but the program behaves correctly anyway –Take care of race conditions in the program Explict programming –Prevent race conditions: typically using locks

9 MPI: deadlocks Deadlocks –Two processes issue receive, waiting for messages from each other, before sending their own message –Again, this simple setting is easy to see, but real programs may have more complex situations Strategies: –Move sends ahead of receives whenever possible –Use asynchronous receives irecv: user specifies a buffer in which to recv, and goes on Check from time to time if the recv has completed Wait for all pending recvs when you can’t go on

10 MPI: buffer overflows MPI send and receives are not synchronized across processors –I.e. sender doesn’t wait until recv is executed So, data (message) must be buffered until it is received –May lead to buffer overflows typically, crashes the program –These are time dependent: In some runs, the receiver may be fast and empty the buffers in time –Can’t allow this in critical, real-time application Solutions: –guarantee adequate buffer size Analysis MPI implementations allow specification of buffer sizes –Synchronized data exchange

11 Charm++ Asynchronous model implies that deadlocks are not possible –There is no waiting statement (no recv, or lock..) But asynchronous behavior means you cannot make assumptions about when things will happen –Object A sends a method invocation to object B, then prints “A” –Object B prints “B” when it executes the method –The two prints may happen in any order

12 Charm++ Example: –Object A sends to messages (method invocations) to B –They may arrive at B (and executed) in different order Another example: –You contribute to a reduction, and then send a message to a neighboring chare –There is no particular order in which you can expect the reduction results and and the neighbor message will arrive: must program for both possibilities Solution strategy: –Program for different message orders