1. CS4231 Parallel and Distributed Algorithms AY 2006/2007 Semester 2 Lecture 1 (12/01/2006) Instructor: Haifeng YU

2. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 22 Course Oveview  Course homepage: go to IVLE and search for CS4231  Check course homepage often!  Feedbacks strongly encouraged  Prerequisite:  (CS1102 || CS1102C || CS1102S) && CS2105  What is the course about:  Designing parallel/distributed algorithms and proving their correctness/properties  If you hated CS1102, you may dislike this course as well…

3. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 23 Why This Course Is Important  This course is more theoretical than other courses  Somewhat similar to CS1102  Theoretical = impractical = useless?  No. Actually it directly relates to your career…  Reason #1: Deep understanding of distributed algorithms distinguishes you from others  (Unfortunately) Everyone’s career has a lot to do with competing with other people  Good programming skills ensure that you don’t lose to others; deep understanding helps you to win

4. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 24 Why This Course Is Important  Reason #2: Understanding foundational concepts enables you to learn new material in the future  Giving you some meat vs. giving you a hunting gun  Computer science is particularly fast-changing – don’t expect what you learn today can be directly applicable 10 years later  Reason #3: Foundational concepts are harder to grasp  This can be your only opportunity to learn these concepts in your whole career

5. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 25 An Example  Your boss asks you to do the following simple protocol  Two nodes A and B  Each has a starting value of 0 or 1  They can communicate but messages can be lost  Goal: A and B should decide on the same value. If A and B both start with 0, the decision should be 0. If A and B both start with 1 and no messages are lost, the decision should be 1.

6. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 26 Relation with CS3211  CS3211  PARALLEL AND CONCURRENT PROGRAMMING  CS3211 is more programming oriented  CS4231 is more about designing algorithms and proving their correctness/properties  Analogy: Basic programming vs. data structure  You will get more from both courses if you take both  Overlap is minimum

7. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 27 Is This Course Hard?  4000-level course: Not usually recommended for freshman/sophomore  It is not a course that you can pass without spending time – not a leisure course  My guideline: 10 hours workload / week (average only)  On the other hand, you should not need to worry about passing if:  You attend all lectures and understand the lecture  You do all homework assignments and understand them  Get questions answered in time  Basically, if you do what you are supposed to do, you’ll have no problem passing  Of course, I hope your goal is higher

8. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 28 Course Format  Instructor: Haifeng YU (office S15-04-08)  Required text book:  “Concurrent and Distributed Computing in Java” by Vijay Garg  Don’t just borrow one…  Go to http://www.ece.utexas.edu/~garg/dist/jbk-corrections.txt  Teaching format:  Friday 1-3pm: Office hour  Friday 3-6pm: Lecturing + tutorial for exercises  Read material before each lecture (~3 hours)  You won’t follow if you don’t read beforehand  Homework (~4 hours)  No programming – to avoid overlapping CS3211

9. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 29 Grading Policy  40% middle-term and 50% final exam  Both exams cover whatever have been taught by the time of the exam  Homework assignments will be marked but grade does not directly contribute to final score  BUT some exam questions will be variants of homework questions  10% class participation – 10 points total  Among other things: -1 if miss a homework assignment  Cheating ABSOLUTELY not tolerated

10. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 210 Material Covered Today and Next Week  Today: Chapter 2 “Mutual Exclusion Problem”  No tutorial today  Next week:  Chapter 3 “Synchronization Primitives”  Read before you come to class next week

11. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 211 Break

12. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 212 Mutual Exclusion Problem  Context: Shared memory systems  Multi-processor computers  Multi-threaded programs  Shared variable x  Initial value 0  Program x = x+1 process 0process 1 read x into a register (value read: 0) increment the register (1) write value in register back to x (1) read x into a register (value read: 1) increment the register (2) write value in register back to x (2)

13. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 213 Mutual Exclusion Problem  Context: Shared memory systems  Multi-processor computers  Multi-threaded programs  Shared variable x  Initial value 0  Program x = x+1 process 0process 1 read x into a register (value read: 0) increment the register (1) read x into a register (value read: 0) increment the register (1) write value in register back to x (1)

14. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 214 Critical Section RequestCS(int processId) Read x into a register Increment the register Write the register value back to x ReleaseCS(int processId) Critical Section

15. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 215 Roadmap  Software solutions  Unsuccessful attempts  Peterson’s algorithm  Bakery algorithm  Hardware solutions  Disabling interrupts to prevent context switch  Special machine-level instructions

16. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 216 Attempt 1 Shared boolean variable openDoor; //whether door is open RequestCS(int processId) { while (openDoor == false) {}; openDoor = false; // close door behind me } ReleaseCS(int processId) { openDoor = true; // open door to let other people in } Both process may see openDoor as true Violate mutual exclusion: Two processes in critical region

17. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 217 Attempt 2 Process 0 RequestCS(0) { wantCS[0] = true; while (wantCS[1] == true) {}; } ReleaseCS(0) { wantCS[0] = false; } Shared boolean variable wantCS[0], wantCS[1] initialized to false Process 1 RequestCS(1) { wantCS[1] = true; while (wantCS[0] == true) {}; } ReleaseCS(1) { wantCS[1] = false; } wantCS[0] = wantCS[1] = true No progress: No one can enter critical region

18. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 218 Attempt 3 Shared int turn initialized to 0 Process 0 RequestCS(0) { while (turn == 1) {}; } ReleaseCS(0) { turn = 1; } Process 1 RequestCS(1) { while (turn == 0) {}; } ReleaseCS(1) { turn = 0; } Starvation: Process 0 may never enter critical region again

19. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 219 Properties Needed  Mutual exclusion: No more than one process in the critical section  Progress: Some one can enter the critical  No starvation: Eventually a process can enter the critical section

20. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 220 Peterson’s Algorithm Shared bool wantCS[0] = false, bool wantCS[1] = false, int turn = 0; Process 0 RequestCS(0) { wantCS[0] = true; turn = 1; while (wantCS[1] == true && turn == 1) {}; } ReleaseCS(0) { wantCS[0] = false; } Process 1 RequestCS(1) { wantCS[1] = true; turn = 0; while (wantCS[0] == true && turn == 0) {}; } ReleaseCS(1) { wantCS[1] = false; }

21. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 221 Correctness Proof for Peterson’s Alg. Process 0 RequestCS(0) { wantCS[0] = true; turn = 1; while (wantCS[1] == true && turn == 1) {}; } Process 1 RequestCS(1) { wantCS[1] = true; turn = 0; while (wantCS[0] == true && turn == 0) {}; } Mutual exclusion: Proof by contradiction. W.l.o.g, assume turn == 0. Then wantCS[0] == false as seen by Process 1. But wantCS[0] set to true by Process 0.

22. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 222 Correctness Proof for Peterson’s Alg. Process 0 RequestCS(0) { wantCS[0] = true; turn = 1; while (wantCS[1] == true && turn == 1) {}; } Process 1 RequestCS(1) { wantCS[1] = true; turn = 0; while (wantCS[0] == true && turn == 0) {}; } Progress: Proof by contradiction. W.l.o.g, assume turn == 0. Done.

23. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 223 Correctness Proof for Peterson’s Alg. Process 0 RequestCS(0) { wantCS[0] = true; turn = 1; while (wantCS[1] == true && turn == 1) {}; } Process 1 RequestCS(1) { wantCS[1] = true; turn = 0; while (wantCS[0] == true && turn == 0) {}; } ReleaseCS(1) { wantCS[1] = false; } No starvation: If Process 0 waiting, then wantCS[1] = true and turn = 1. Process 1 in critical region -- will exit and set wantCS[1] to false.

24. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 224 Lamport’s Bakery Alg.  For n processes  Get a number first  Get served when all people with lower number have been served  Two shared arrays of n elements  boolean choosing[i] = false; // process i is trying to get a number  int number[i] = 0; // the number got by process i; “0” means process i not interested in being served

25. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 225 ReleaseCS(int myid) { number[myid] = 0; } // a utility function boolean Smaller(int number1, int id1, int number2, int id2) { if (number1 < number 2) return true; if (number1 == number2) { if (id1 < id2) return true; else return false; } if (number 1 > number2) return false; }

26. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 226 RequestCS(int myid) { choosing[myid] = true; for (int j = 0; j < n; j++) if (number[j] > number[myid]) number[myid] = number[j]; number[myid]++; choosing[myid] = false; for (int j = 0; j < N; j++) { while (choosing[j] == true) {}; while (number[j] != 0 && Smaller(number[j], j, number[myid], myid)) {}; } get a number wait for people ahead of me

27. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 227 choosing[myid] = true; for (int j = 0; j < n; j++) if (number[j] > number[myid]) number[myid] = number[j]; number[myid]++; choosing[myid] = false; for (int j = 0; j < n; j++) { while (choosing[j] == true) {}; while (number[j] != 0 && Smaller(number[j], j, number[myid], myid)) {}; }  Progress:  Proof by contradiction. Suppose i and k both blocked.  Case 1:  Case 2: smaller(number[i ], i, number[k], k) and smaller (number[k], k, number[i ], i) Impossible due to the tie-break  No starvation is trivial to show

28. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 228 choosing[myid] = true; for (int j = 0; j < n; j++) if (number[j] > number[myid]) number[myid] = number[j]; number[myid]++; choosing[myid] = false; for (int j = 0; j < n; j++) { while (choosing[j] == true) {}; while (number[j] != 0 && Smaller(number[j], j, number[myid], myid)) {}; }  Mutual exclusion:  Proof by contradiction. Suppose i and k both in critical section.  W.l.o.g, assume Smaller(number[i ], i, number[k], k).  Process k must have invoked this when number[i ] == 0.  Case 1: Process i has not executed “if (number[k] > number[i ])”  Case 2: Has executed “if ()”  Subcase 2-1: Process i has executed “choosing[i ] = false” - impossible since number[i ] == 0  Subcase 2-2: Has not executed “choosing[i ] = false”

29. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 229 choosing[myid] = true; for (int j = 0; j < n; j++) if (number[j] > number[myid]) number[myid] = number[j]; number[myid]++; choosing[myid] = false; for (int j = 0; j < n; j++) { while (choosing[j] == true) {}; while (number[j] != 0 && Smaller(number[j], j, number[myid], myid)) {}; }  Carry-over from last slide  Process k must have invoked “Smaller(number[i ], i, number[k], k)” when number[i ] == 0.  Case 2: Process i Has executed “if (number[k] > number[i ])”  Subcase 2-2: Has not executed “choosing[i ] = false”  Process k must have invoked “while (Choosing[i ] = true) {}; before process i starts to choose a number  number[i ] will be larger than number[k]  All cases lead to contradition

30. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 230 Roadmap  Software solutions  Unsuccessful attempts  Peterson’s algorithm  Bakery algorithm  Hardware solutions  Disabling interrupts to prevent context switch  Special machine-level instructions

31. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 231 Disable Interrupts process 1process 2 read x into a register (value read: 0) increment the register (1) read x into a register (value read: 0) increment the register (1) write value in register back to x (1) Do not allow context switch here

32. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 232 Special Machine-level Instructions Shared bool variable openDoor initialized to true; TestAndSet(boolean newValue) { boolean tmp = openDoor; openDoor = newValue; return tmp; } RequestCS(process_id) { while (TestAndSet(false) == false) {}; } ReleaseCS(process_id) { openDoor = true; } Executed atomically – cannot be interrupted

33. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 233 Summary  Course overview & policy  The mutual exclusion problem in shared-memory systems  Software solutions  Unsuccessful attempts  Peterson’s algorithm  Bakery algorithm  Hardware solutions  Disabling interrupts to prevent context switch  Special machine-level instructions

34. CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 234 Homework Assignment  Devise a mutual exclusion algorithm for n processes by using Peterson’s 2-mutual-exclusion algorithm  Page 28:  Problem 2.1 (clearly write out a scenario for 2 processes where problem occur as on slide 13)  Problem 2.3 (same as above)  Problem 2.4 (either give a proof or construct a problematic scenario as above. Do this for all three properties.)  Homework due a week from today  Read Chapter 3